Skip to content

Commit fd62dbc

Browse files
magreenbaumantonbabenko
magreenbaum
and
antonbabenko
authored
feat: Adding analytics configuration support (#193)
Co-authored-by: magreenbaum <magreenbaum> Co-authored-by: Anton Babenko <anton@antonbabenko.com>
1 parent 6fa011f commit fd62dbc

File tree

9 files changed

+332
-12
lines changed

9 files changed

+332
-12
lines changed

‎README.md‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -137,6 +137,7 @@ No modules.
137137
| [aws_s3_bucket.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) | resource |
138138
| [aws_s3_bucket_accelerate_configuration.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_accelerate_configuration) | resource |
139139
| [aws_s3_bucket_acl.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_acl) | resource |
140+
| [aws_s3_bucket_analytics_configuration.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_analytics_configuration) | resource |
140141
| [aws_s3_bucket_cors_configuration.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_cors_configuration) | resource |
141142
| [aws_s3_bucket_intelligent_tiering_configuration.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_intelligent_tiering_configuration) | resource |
142143
| [aws_s3_bucket_inventory.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_inventory) | resource |
@@ -158,7 +159,7 @@ No modules.
158159
| [aws_iam_policy_document.combined](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
159160
| [aws_iam_policy_document.deny_insecure_transport](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
160161
| [aws_iam_policy_document.elb_log_delivery](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
161-
| [aws_iam_policy_document.inventory_destination_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
162+
| [aws_iam_policy_document.inventory_and_analytics_destination_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
162163
| [aws_iam_policy_document.lb_log_delivery](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
163164
| [aws_iam_policy_document.require_latest_tls](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
164165

@@ -168,6 +169,11 @@ No modules.
168169
|------|-------------|------|---------|:--------:|
169170
| <a name="input_acceleration_status"></a> [acceleration\_status](#input\_acceleration\_status) | (Optional) Sets the accelerate configuration of an existing bucket. Can be Enabled or Suspended. | `string` | `null` | no |
170171
| <a name="input_acl"></a> [acl](#input\_acl) | (Optional) The canned ACL to apply. Conflicts with `grant` | `string` | `null` | no |
172+
| <a name="input_analytics_configuration"></a> [analytics\_configuration](#input\_analytics\_configuration) | Map containing bucket analytics configuration. | `any` | `{}` | no |
173+
| <a name="input_analytics_self_source_destination"></a> [analytics\_self\_source\_destination](#input\_analytics\_self\_source\_destination) | Whether or not the analytics source bucket is also the destination bucket. | `bool` | `false` | no |
174+
| <a name="input_analytics_source_account_id"></a> [analytics\_source\_account\_id](#input\_analytics\_source\_account\_id) | The analytics source account id. | `string` | `null` | no |
175+
| <a name="input_analytics_source_bucket_arn"></a> [analytics\_source\_bucket\_arn](#input\_analytics\_source\_bucket\_arn) | The analytics source bucket ARN. | `string` | `null` | no |
176+
| <a name="input_attach_analytics_destination_policy"></a> [attach\_analytics\_destination\_policy](#input\_attach\_analytics\_destination\_policy) | Controls if S3 bucket should have bucket analytics destination policy attached. | `bool` | `false` | no |
171177
| <a name="input_attach_deny_insecure_transport_policy"></a> [attach\_deny\_insecure\_transport\_policy](#input\_attach\_deny\_insecure\_transport\_policy) | Controls if S3 bucket should have deny non-SSL transport policy attached | `bool` | `false` | no |
172178
| <a name="input_attach_elb_log_delivery_policy"></a> [attach\_elb\_log\_delivery\_policy](#input\_attach\_elb\_log\_delivery\_policy) | Controls if S3 bucket should have ELB log delivery policy attached | `bool` | `false` | no |
173179
| <a name="input_attach_inventory_destination_policy"></a> [attach\_inventory\_destination\_policy](#input\_attach\_inventory\_destination\_policy) | Controls if S3 bucket should have bucket inventory destination policy attached. | `bool` | `false` | no |

‎examples/s3-analytics/README.md‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# S3 bucket Analytics Configurations
2+
3+
Configuration in this directory creates an S3 bucket with several analytics configurations including a different destination for analytics reports generated.
4+
5+
Please check [complete example](https://github.com/terraform-aws-modules/terraform-aws-s3-bucket/tree/master/examples/complete) to see all other features supported by this module.
6+
7+
<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
8+
## Requirements
9+
10+
| Name | Version |
11+
|------|---------|
12+
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 0.13.1 |
13+
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 4.9 |
14+
| <a name="requirement_random"></a> [random](#requirement\_random) | >= 2.0 |
15+
16+
## Providers
17+
18+
| Name | Version |
19+
|------|---------|
20+
| <a name="provider_aws"></a> [aws](#provider\_aws) | >= 4.9 |
21+
| <a name="provider_random"></a> [random](#provider\_random) | >= 2.0 |
22+
23+
## Modules
24+
25+
| Name | Source | Version |
26+
|------|--------|---------|
27+
| <a name="module_analytics_and_inventory_destination_bucket"></a> [analytics\_and\_inventory\_destination\_bucket](#module\_analytics\_and\_inventory\_destination\_bucket) | ../../ | n/a |
28+
| <a name="module_analytics_configuration_bucket"></a> [analytics\_configuration\_bucket](#module\_analytics\_configuration\_bucket) | ../../ | n/a |
29+
| <a name="module_analytics_destination_bucket"></a> [analytics\_destination\_bucket](#module\_analytics\_destination\_bucket) | ../../ | n/a |
30+
| <a name="module_inventory_source_bucket"></a> [inventory\_source\_bucket](#module\_inventory\_source\_bucket) | ../.. | n/a |
31+
32+
## Resources
33+
34+
| Name | Type |
35+
|------|------|
36+
| [random_pet.this](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) | resource |
37+
| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
38+
39+
## Inputs
40+
41+
No inputs.
42+
43+
## Outputs
44+
45+
| Name | Description |
46+
|------|-------------|
47+
| <a name="output_s3_bucket_arn"></a> [s3\_bucket\_arn](#output\_s3\_bucket\_arn) | The ARN of the bucket. Will be of format arn:aws:s3:::bucketname. |
48+
| <a name="output_s3_bucket_bucket_domain_name"></a> [s3\_bucket\_bucket\_domain\_name](#output\_s3\_bucket\_bucket\_domain\_name) | The bucket domain name. Will be of format bucketname.s3.amazonaws.com. |
49+
| <a name="output_s3_bucket_bucket_regional_domain_name"></a> [s3\_bucket\_bucket\_regional\_domain\_name](#output\_s3\_bucket\_bucket\_regional\_domain\_name) | The bucket region-specific domain name. The bucket domain name including the region name, please refer here for format. Note: The AWS CloudFront allows specifying S3 region-specific endpoint when creating S3 origin, it will prevent redirect issues from CloudFront to S3 Origin URL. |
50+
| <a name="output_s3_bucket_hosted_zone_id"></a> [s3\_bucket\_hosted\_zone\_id](#output\_s3\_bucket\_hosted\_zone\_id) | The Route 53 Hosted Zone ID for this bucket's region. |
51+
| <a name="output_s3_bucket_id"></a> [s3\_bucket\_id](#output\_s3\_bucket\_id) | The name of the bucket. |
52+
| <a name="output_s3_bucket_region"></a> [s3\_bucket\_region](#output\_s3\_bucket\_region) | The AWS region this bucket resides in. |
53+
| <a name="output_s3_bucket_website_domain"></a> [s3\_bucket\_website\_domain](#output\_s3\_bucket\_website\_domain) | The domain of the website endpoint, if the bucket is configured with a website. If not, this will be an empty string. This is used to create Route 53 alias records. |
54+
| <a name="output_s3_bucket_website_endpoint"></a> [s3\_bucket\_website\_endpoint](#output\_s3\_bucket\_website\_endpoint) | The website endpoint, if the bucket is configured with a website. If not, this will be an empty string. |
55+
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

‎examples/s3-analytics/main.tf‎

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
locals {
2+
bucket_name = "s3-bucket-${random_pet.this.id}"
3+
region = "eu-west-1"
4+
}
5+
6+
provider "aws" {
7+
region = local.region
8+
9+
# Make it faster by skipping something
10+
skip_metadata_api_check = true
11+
skip_region_validation = true
12+
skip_credentials_validation = true
13+
skip_requesting_account_id = true
14+
}
15+
16+
data "aws_caller_identity" "current" {}
17+
18+
module "analytics_configuration_bucket" {
19+
source = "../../"
20+
21+
bucket = local.bucket_name
22+
23+
force_destroy = true
24+
25+
attach_analytics_destination_policy = true
26+
attach_policy = true
27+
analytics_self_source_destination = true
28+
acl = "private" # "acl" conflicts with "grant" and "owner"
29+
30+
versioning = {
31+
status = true
32+
mfa_delete = false
33+
}
34+
35+
analytics_configuration = {
36+
37+
# No exporting
38+
prefix_documents = {
39+
filter = {
40+
prefix = "documents/"
41+
}
42+
}
43+
44+
# Same source and destination bucket
45+
tags = {
46+
filter = {
47+
tags = {
48+
production = "true"
49+
}
50+
}
51+
storage_class_analysis = {
52+
output_schema_version = "V_1"
53+
}
54+
}
55+
56+
# Different destination bucket
57+
all = {
58+
storage_class_analysis = {
59+
destination_bucket_arn = module.analytics_destination_bucket.s3_bucket_arn
60+
prefix = "analytics"
61+
}
62+
}
63+
64+
# Different destination shared with inventory destination
65+
example = {
66+
storage_class_analysis = {
67+
destination_bucket_arn = module.analytics_and_inventory_destination_bucket.s3_bucket_arn
68+
}
69+
}
70+
}
71+
}
72+
73+
resource "random_pet" "this" {
74+
length = 2
75+
}
76+
77+
module "analytics_destination_bucket" {
78+
source = "../../"
79+
80+
bucket = "analytics-destination-${random_pet.this.id}"
81+
acl = "private" # "acl" conflicts with "grant" and "owner"
82+
force_destroy = true
83+
attach_policy = true
84+
attach_analytics_destination_policy = true
85+
analytics_source_bucket_arn = module.analytics_configuration_bucket.s3_bucket_arn
86+
analytics_source_account_id = data.aws_caller_identity.current.id
87+
}
88+
89+
# Inventory configuration for shared destination example
90+
module "inventory_source_bucket" {
91+
source = "../.."
92+
93+
bucket = "inventory-source-${random_pet.this.id}"
94+
95+
force_destroy = true
96+
acl = "private" # "acl" conflicts with "grant" and "owner"
97+
98+
inventory_configuration = {
99+
destination_other = {
100+
included_object_versions = "All"
101+
destination = {
102+
bucket_arn = module.analytics_and_inventory_destination_bucket.s3_bucket_arn
103+
format = "CSV"
104+
encryption = {
105+
encryption_type = "sse_s3"
106+
}
107+
}
108+
frequency = "Daily"
109+
optional_fields = ["Size", "EncryptionStatus", "StorageClass", "ChecksumAlgorithm"]
110+
}
111+
}
112+
}
113+
114+
# Example of using the same destination bucket for analytics and inventory
115+
module "analytics_and_inventory_destination_bucket" {
116+
source = "../../"
117+
118+
bucket = "analytics-and-inventory-destination-${random_pet.this.id}"
119+
acl = "private" # "acl" conflicts with "grant" and "owner"
120+
force_destroy = true
121+
attach_policy = true
122+
123+
# Analytics bucket policy settings
124+
attach_analytics_destination_policy = true
125+
analytics_source_bucket_arn = module.analytics_configuration_bucket.s3_bucket_arn
126+
analytics_source_account_id = data.aws_caller_identity.current.id
127+
128+
# Inventory bucket policy settings
129+
attach_inventory_destination_policy = true
130+
inventory_source_bucket_arn = module.inventory_source_bucket.s3_bucket_arn
131+
inventory_source_account_id = data.aws_caller_identity.current.id
132+
}

‎examples/s3-analytics/outputs.tf‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
output "s3_bucket_id" {
2+
description = "The name of the bucket."
3+
value = module.analytics_configuration_bucket.s3_bucket_id
4+
}
5+
6+
output "s3_bucket_arn" {
7+
description = "The ARN of the bucket. Will be of format arn:aws:s3:::bucketname."
8+
value = module.analytics_configuration_bucket.s3_bucket_arn
9+
}
10+
11+
output "s3_bucket_bucket_domain_name" {
12+
description = "The bucket domain name. Will be of format bucketname.s3.amazonaws.com."
13+
value = module.analytics_configuration_bucket.s3_bucket_bucket_domain_name
14+
}
15+
16+
output "s3_bucket_bucket_regional_domain_name" {
17+
description = "The bucket region-specific domain name. The bucket domain name including the region name, please refer here for format. Note: The AWS CloudFront allows specifying S3 region-specific endpoint when creating S3 origin, it will prevent redirect issues from CloudFront to S3 Origin URL."
18+
value = module.analytics_configuration_bucket.s3_bucket_bucket_regional_domain_name
19+
}
20+
21+
output "s3_bucket_hosted_zone_id" {
22+
description = "The Route 53 Hosted Zone ID for this bucket's region."
23+
value = module.analytics_configuration_bucket.s3_bucket_hosted_zone_id
24+
}
25+
26+
output "s3_bucket_region" {
27+
description = "The AWS region this bucket resides in."
28+
value = module.analytics_configuration_bucket.s3_bucket_region
29+
}
30+
31+
output "s3_bucket_website_endpoint" {
32+
description = "The website endpoint, if the bucket is configured with a website. If not, this will be an empty string."
33+
value = module.analytics_configuration_bucket.s3_bucket_website_endpoint
34+
}
35+
36+
output "s3_bucket_website_domain" {
37+
description = "The domain of the website endpoint, if the bucket is configured with a website. If not, this will be an empty string. This is used to create Route 53 alias records. "
38+
value = module.analytics_configuration_bucket.s3_bucket_website_domain
39+
}

‎examples/s3-analytics/variables.tf‎

Whitespace-only changes.

‎examples/s3-analytics/versions.tf‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
terraform {
2+
required_version = ">= 0.13.1"
3+
4+
required_providers {
5+
aws = {
6+
source = "hashicorp/aws"
7+
version = ">= 4.9"
8+
}
9+
random = {
10+
source = "hashicorp/random"
11+
version = ">= 2.0"
12+
}
13+
}
14+
}

‎main.tf‎

Lines changed: 50 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -518,7 +518,7 @@ data "aws_iam_policy_document" "combined" {
518518
var.attach_lb_log_delivery_policy ? data.aws_iam_policy_document.lb_log_delivery[0].json : "",
519519
var.attach_require_latest_tls_policy ? data.aws_iam_policy_document.require_latest_tls[0].json : "",
520520
var.attach_deny_insecure_transport_policy ? data.aws_iam_policy_document.deny_insecure_transport[0].json : "",
521-
var.attach_inventory_destination_policy ? data.aws_iam_policy_document.inventory_destination_policy[0].json : "",
521+
var.attach_inventory_destination_policy || var.attach_analytics_destination_policy ? data.aws_iam_policy_document.inventory_and_analytics_destination_policy[0].json : "",
522522
var.attach_policy ? var.policy : ""
523523
])
524524
}
@@ -792,13 +792,13 @@ resource "aws_s3_bucket_inventory" "this" {
792792
}
793793
}
794794

795-
# Inventory destination bucket requires a bucket policy to allow source to PutObjects
795+
# Inventory and analytics destination bucket requires a bucket policy to allow source to PutObjects
796796
# https://docs.aws.amazon.com/AmazonS3/latest/userguide/example-bucket-policies.html#example-bucket-policies-use-case-9
797-
data "aws_iam_policy_document" "inventory_destination_policy" {
798-
count = local.create_bucket && var.attach_inventory_destination_policy ? 1 : 0
797+
data "aws_iam_policy_document" "inventory_and_analytics_destination_policy" {
798+
count = local.create_bucket && var.attach_inventory_destination_policy || var.attach_analytics_destination_policy ? 1 : 0
799799

800800
statement {
801-
sid = "destinationInventoryPolicy"
801+
sid = "destinationInventoryAndAnalyticsPolicy"
802802
effect = "Allow"
803803

804804
actions = [
@@ -817,16 +817,18 @@ data "aws_iam_policy_document" "inventory_destination_policy" {
817817
condition {
818818
test = "ArnLike"
819819
variable = "aws:SourceArn"
820-
values = [
821-
var.inventory_self_source_destination ? aws_s3_bucket.this[0].arn : var.inventory_source_bucket_arn
822-
]
820+
values = compact(distinct([
821+
var.inventory_self_source_destination ? aws_s3_bucket.this[0].arn : var.inventory_source_bucket_arn,
822+
var.analytics_self_source_destination ? aws_s3_bucket.this[0].arn : var.analytics_source_bucket_arn
823+
]))
823824
}
824825

825826
condition {
826827
test = "StringEquals"
827-
values = [
828-
var.inventory_self_source_destination ? data.aws_caller_identity.current.id : var.inventory_source_account_id
829-
]
828+
values = compact(distinct([
829+
var.inventory_self_source_destination ? data.aws_caller_identity.current.id : var.inventory_source_account_id,
830+
var.analytics_self_source_destination ? data.aws_caller_identity.current.id : var.analytics_source_account_id
831+
]))
830832
variable = "aws:SourceAccount"
831833
}
832834

@@ -837,3 +839,40 @@ data "aws_iam_policy_document" "inventory_destination_policy" {
837839
}
838840
}
839841
}
842+
843+
resource "aws_s3_bucket_analytics_configuration" "this" {
844+
for_each = { for k, v in var.analytics_configuration : k => v if local.create_bucket }
845+
846+
bucket = aws_s3_bucket.this[0].id
847+
name = each.key
848+
849+
dynamic "filter" {
850+
for_each = length(try(flatten([each.value.filter]), [])) == 0 ? [] : [true]
851+
852+
content {
853+
prefix = try(each.value.filter.prefix, null)
854+
tags = try(each.value.filter.tags, null)
855+
}
856+
}
857+
858+
dynamic "storage_class_analysis" {
859+
for_each = length(try(flatten([each.value.storage_class_analysis]), [])) == 0 ? [] : [true]
860+
861+
content {
862+
863+
data_export {
864+
output_schema_version = try(each.value.storage_class_analysis.output_schema_version, null)
865+
866+
destination {
867+
868+
s3_bucket_destination {
869+
bucket_arn = try(each.value.storage_class_analysis.destination_bucket_arn, aws_s3_bucket.this[0].arn)
870+
bucket_account_id = try(each.value.storage_class_analysis.destination_account_id, data.aws_caller_identity.current.id)
871+
format = try(each.value.storage_class_analysis.export_format, "CSV")
872+
prefix = try(each.value.storage_class_analysis.export_prefix, null)
873+
}
874+
}
875+
}
876+
}
877+
}
878+
}

0 commit comments

Comments
 (0)