macie updates to handle API errors, default values

Author: andywick-aws

CHANGELOG.md

@@ -3,7 +3,7 @@
 ## Table of Contents<!-- omit in toc -->
 
 - [Introduction](#introduction)
-- [2022-07-12](#2022-07-12)
+- [2022-07-15](#2022-07-15)
 - [2022-05-23](#2022-05-23)
 - [2022-05-15](#2022-05-15)
 - [2022-04-25](#2022-04-25)
@@ -30,14 +30,17 @@ All notable changes to this project will be documented in this file.
 
 ---
 
-## 2022-07-12
+## 2022-07-15
 
 ### Changed<!-- omit in toc -->
 
 - Added Checkov Lambda Function suppressions for CKV_AWS_115 (Reserved Concurrent Executions) and CKV_AWS_117 (Run within a VPC) to all solution templates with Lambda Function configurations.
 - Updated Lambda python files to fix mypy finding for log_level to always be a string value.
 - Updated the [customizations-for-aws-control-tower.template](aws_sra_examples/solutions/common/common_cfct_setup/templates/customizations-for-aws-control-tower.template) to the latest version v2.4.0 and added Checkov suppressions.
 - Updated pyproject.toml dependencies to the latest versions.
+- Updated [Macie](aws_sra_examples/solutions/macie/macie_org) solution to handle API errors when creating existing members.
+- Updated [EC2 Default EBS Encryption](aws_sra_examples/solutions/ec2/ec2_default_ebs_encryption) to include default string value for the pExcludeEC2DefaultEBSEncryptionTags parameter.
+- Updated [Account Alternate Contacts](aws_sra_examples/solutions/account/account_alternate_contacts) to include default string value for the pExcludeAlternateContactAccountTags parameter.
 
 ## 2022-05-23
 
@@ -127,7 +130,6 @@ All notable changes to this project will be documented in this file.
 ### Removed<!-- omit in toc -->
 
 - Removed the sra-common-cfct-setup-main-ssm.yaml template as it was the same as the other main template.
-  > > > > > > > 035d75801d00b0f08affe2bf91d7cbfeade1820f
 
 ## 2022-03-16
 

aws_sra_examples/solutions/account/account_alternate_contacts/templates/sra-account-alternate-contacts-main-ssm.yaml

@@ -169,6 +169,7 @@ Parameters:
     Type: String
   pExcludeAlternateContactAccountTags:
     AllowedPattern: '^$|.*'
+    Default: ''
     Description:
       '(Optional) Resource Tags that denote an Account should be excluded from this solution in JSON format: [{"Key": "string", "Value": "string"},
       ... ]. For example, [{"Key": "exclude-alternate-contacts", "Value": "true"}].'

aws_sra_examples/solutions/account/account_alternate_contacts/templates/sra-account-alternate-contacts-main.yaml

@@ -168,6 +168,7 @@ Parameters:
     Type: String
   pExcludeAlternateContactAccountTags:
     AllowedPattern: '^$|.*'
+    Default: ''
     Description:
       '(Optional) Resource Tags that denote an Account should be excluded from this solution in JSON format: [{"Key": "string", "Value": "string"},
       ... ]. For example, [{"Key": "exclude-alternate-contacts", "Value": "true"}].'

aws_sra_examples/solutions/ec2/ec2_default_ebs_encryption/templates/sra-ec2-default-ebs-encryption-main-ssm.yaml

@@ -106,6 +106,7 @@ Parameters:
     Type: String
   pExcludeEC2DefaultEBSEncryptionTags:
     AllowedPattern: '^$|.*'
+    Default: ''
     Description:
       '(Optional) Resource Tags that denote an Account should be excluded from this solution in JSON format: [{"Key": "string", "Value": "string"},
       ... ]. For example, [{"Key": "exclude-ec2-default-ebs-encryption", "Value": "true"}].'

aws_sra_examples/solutions/ec2/ec2_default_ebs_encryption/templates/sra-ec2-default-ebs-encryption-main.yaml

@@ -106,6 +106,7 @@ Parameters:
     Type: String
   pExcludeEC2DefaultEBSEncryptionTags:
     AllowedPattern: '^$|.*'
+    Default: ''
     Description:
       '(Optional) Resource Tags that denote an Account should be excluded from this solution in JSON format: [{"Key": "string", "Value": "string"},
       ... ]. For example, [{"Key": "exclude-ec2-default-ebs-encryption", "Value": "true"}].'

aws_sra_examples/solutions/macie/macie_org/lambda/src/app.py

@@ -5,7 +5,7 @@
 - enables new accounts automatically
 - publishes findings to an S3 bucket
 
-Version: 1.1
+Version: 1.2
 
 'macie_org' solution in the repo, https://github.com/aws-samples/aws-security-reference-architecture-examples
 
@@ -81,6 +81,14 @@ def process_create_update_event(params: dict, regions: list) -> None:
             params.get("CONFIGURATION_ROLE_NAME", ""), "sra-macie-org", params.get("DELEGATED_ADMIN_ACCOUNT_ID", "")
         )
 
+        LOGGER.info("Enabling Macie in the Management Account")
+        macie.enable_macie(
+            params["MANAGEMENT_ACCOUNT_ID"],
+            "",
+            regions,
+            params["FINDING_PUBLISHING_FREQUENCY"],
+        )
+
         macie.configure_macie(
             delegated_admin_session,
             params["DELEGATED_ADMIN_ACCOUNT_ID"],
@@ -143,6 +151,7 @@ def get_validated_parameters(event: CloudFormationCustomResourceEvent) -> dict:
         params.get("SNS_TOPIC_ARN", ""),
         pattern=r"^arn:(aws[a-zA-Z-]*){1}:sns:[a-z0-9-]+:\d{12}:[0-9a-zA-Z]+([0-9a-zA-Z-]*[0-9a-zA-Z])*$",
     )
+    parameter_pattern_validator("MANAGEMENT_ACCOUNT_ID", params.get("MANAGEMENT_ACCOUNT_ID", ""), pattern=r"^\d{12}$")
 
     return params
 
@@ -188,11 +197,7 @@ def process_cloudformation_event(event: CloudFormationCustomResourceEvent, conte
         account_ids = common.get_account_ids([], params["DELEGATED_ADMIN_ACCOUNT_ID"])
         macie.process_delete_event(params, regions, account_ids, False)
 
-    return (
-        f"sra-macie-{params['DELEGATED_ADMIN_ACCOUNT_ID']}-{params['DISABLE_MACIE_ROLE_NAME']}-{params['CONTROL_TOWER_REGIONS_ONLY']}-"
-        + f"{params['FINDING_PUBLISHING_FREQUENCY']}-{len(params['KMS_KEY_ARN'])}-{params['PUBLISHING_DESTINATION_BUCKET_NAME']}-"
-        + f"{len(params['SNS_TOPIC_ARN'])}"
-    )
+    return f"sra-macie-{params['DELEGATED_ADMIN_ACCOUNT_ID']}"
 
 
 def lambda_handler(event: Dict[str, Any], context: Context) -> None:

aws_sra_examples/solutions/macie/macie_org/lambda/src/macie.py

@@ -1,6 +1,6 @@
 """This script provides logic for managing Macie.
 
-Version: 1.1
+Version: 1.2
 
 'macie_org' solution in the repo, https://github.com/aws-samples/aws-security-reference-architecture-examples
 
@@ -29,7 +29,7 @@
 
 # Global variables
 SERVICE_NAME = "macie.amazonaws.com"
-SLEEP_SECONDS = 20
+SLEEP_SECONDS = 30
 UNEXPECTED = "Unexpected!"
 
 try:
@@ -94,8 +94,18 @@ def create_members(macie2_client: Macie2Client, accounts: list) -> None:
     """
     LOGGER.info("...Creating members")
     for account in accounts:
-        macie2_client.create_member(account={"accountId": account["AccountId"], "email": account["Email"]})
-        sleep(0.2)  # Sleeping .2 second to avoid max API call error
+        try:
+            create_member_response = macie2_client.create_member(account={"accountId": account["AccountId"], "email": account["Email"]})
+            api_call_details = {"API_Call": "macie2:CreateMember", "API_Response": create_member_response}
+            LOGGER.info(api_call_details)
+            sleep(1)  # Sleeping 1 second to avoid max API call error
+        except ClientError as error:
+            LOGGER.info(f"Error creating member {account['AccountId']} - {error}")
+            LOGGER.info("...Waiting 10 seconds to try adding member again.")
+            sleep(10)  # Wait for delegated admin to get configured
+            create_member_response = macie2_client.create_member(account={"accountId": account["AccountId"], "email": account["Email"]})
+            api_call_details = {"API_Call": "macie2:CreateMember", "API_Response": create_member_response}
+            LOGGER.info(api_call_details)
 
 
 def configure_macie(
@@ -130,13 +140,44 @@ def configure_macie(
         )
 
         # Create members for existing Organization accounts
-        LOGGER.debug(f"Existing Accounts: {accounts}")
+        LOGGER.info(f"Existing Accounts: {accounts}")
         create_members(regional_client, accounts)
 
         # Update Organization configuration to automatically enable new accounts
         regional_client.update_organization_configuration(autoEnable=True)
 
 
+def enable_macie(
+    account_id: str,
+    configuration_role_name: str,
+    regions: list,
+    finding_publishing_frequency: Union[Literal["FIFTEEN_MINUTES"], Literal["ONE_HOUR"], Literal["SIX_HOURS"]],
+) -> None:
+    """Enable Macie with provided parameters.
+
+    Args:
+        account_id: Account ID
+        configuration_role_name: Configuration Role Name (Optional)
+        regions: AWS Region List
+        finding_publishing_frequency: Finding Publishing Frequency
+    """
+    account_session: boto3.Session = boto3.Session()
+
+    if configuration_role_name:
+        account_session = common.assume_role(configuration_role_name, "sra-enable-macie", account_id)
+
+    # Loop through the regions and enable Macie
+    for region in regions:
+        regional_client: Macie2Client = account_session.client("macie2", region_name=region)
+        try:
+            enable_macie_response = regional_client.enable_macie(findingPublishingFrequency=finding_publishing_frequency, status="ENABLED")
+            api_call_details = {"API_Call": "macie2:EnableMacie", "API_Response": enable_macie_response}
+            LOGGER.info(api_call_details)
+            sleep(0.2)  # Sleeping .2 second to avoid max API call error
+        except regional_client.exceptions.ConflictException:
+            LOGGER.info(f"Macie already enabled in {region}.")
+
+
 def process_delete_event(params: dict, regions: list, account_ids: list, include_members: bool = False) -> None:
     """Delete Macie solution resources.
 

aws_sra_examples/solutions/macie/macie_org/templates/sra-macie-org-configuration.yaml

@@ -9,7 +9,7 @@ Description:
 
 Metadata:
   SRA:
-    Version: 1.1
+    Version: 1.2
     Order: 5
   AWS::CloudFormation::Interface:
     ParameterGroups:
@@ -302,6 +302,7 @@ Resources:
                 Effect: Allow
                 Action:
                   - macie2:DisableOrganizationAdminAccount
+                  - macie2:EnableMacie
                   - macie2:EnableOrganizationAdminAccount
                   - macie2:ListOrganizationAdminAccounts
                 Resource: '*'
@@ -423,6 +424,7 @@ Resources:
       ENABLED_REGIONS: !Ref pEnabledRegions
       FINDING_PUBLISHING_FREQUENCY: !Ref pFindingPublishingFrequency
       KMS_KEY_ARN: !Ref pKMSKeyArn
+      MANAGEMENT_ACCOUNT_ID: !Ref AWS::AccountId
       PUBLISHING_DESTINATION_BUCKET_NAME: !Ref pPublishingDestinationBucketName
       SNS_TOPIC_ARN: !Ref rMacieOrgTopic