Merge pull request #38 from taylorludwig/feature/37-terraform-created-files-in-archive

Feature/37 terraform created files in archive

Author: aaron-lane

.gitignore

@@ -51,6 +51,8 @@ credentials.json
 examples/automatic-labelling-folder/function_source.zip
 examples/automatic-labelling-from-localhost/function_source.zip
 examples/automatic-labelling-from-repository/function_source_copy
+examples/dynamic-files/function_source.zip
+examples/dynamic-files/function_source/terraform_created_file.txt
 
 node_modules
 yarn.lock

CHANGELOG.md

@@ -7,6 +7,10 @@ project adheres to [Semantic Versioning](http://semver.org/).
 
 ## [Unreleased]
 
+### Added
+
+- The `source_dependent_files` variable. If used `archive` won't be created until Terraform created `local_file`s are finished. [#38]
+
 ### Fixed
 
 - Updating the source for a local event-function doesn't update the function. [#32]
@@ -36,6 +40,7 @@ project adheres to [Semantic Versioning](http://semver.org/).
 [1.1.0]: https://github.com/terraform-google-modules/terraform-google-event-function/compare/v1.0.0...v1.1.0
 
 
+[#38]: https://github.com/terraform-google-modules/terraform-google-event-function/issues/38
 [#32]: https://github.com/terraform-google-modules/terraform-google-event-function/issues/32
 [#28]: https://github.com/terraform-google-modules/terraform-google-event-function/pull/28
 [#23]: https://github.com/terraform-google-modules/terraform-google-event-function/pull/23

Makefile

@@ -18,7 +18,7 @@
 # Make will use bash instead of sh
 SHELL := /usr/bin/env bash
 
-DOCKER_TAG_VERSION_DEVELOPER_TOOLS := 0.4.3
+DOCKER_TAG_VERSION_DEVELOPER_TOOLS := 0
 DOCKER_IMAGE_DEVELOPER_TOOLS := cft/developer-tools
 REGISTRY_URL := gcr.io/cloud-foundation-cicd
 

README.md

@@ -26,6 +26,26 @@ The
 is a tested reference of how to use the root module with the
 [event-project-log-entry submodule][event-project-log-entry-submodule].
 
+## Terraform Created Source Files
+
+If you have `local_file` Terraform resources that need to be included in the function's archive include them in the optional `source_dependent_files`.
+
+This will tell the module to wait until those files exist before creating the archive.
+
+Example can also be seen in `examples/dynamic-files`
+
+```hcl
+resource "local_file" "file" {
+  content  = "some content"
+  filename = "${path.module}/function_source/terraform_created_file.txt"
+}
+
+module "localhost_function" {
+  ...
+
+  source_dependent_files = [local_file.file]
+}
+```
 <!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
 ## Inputs
 
@@ -46,6 +66,7 @@ is a tested reference of how to use the root module with the
 | region | The region in which resources will be applied. | string | n/a | yes |
 | runtime | The runtime in which the function will be executed. | string | n/a | yes |
 | service\_account\_email | The service account to run the function as. | string | `""` | no |
+| source\_dependent\_files | A list of any Terraform created `local_file`s that the module will wait for before creating the archive. | object | `<list>` | no |
 | source\_directory | The pathname of the directory which contains the function source code. | string | n/a | yes |
 | timeout\_s | The amount of time in seconds allotted for the execution of the function. | number | `"60"` | no |
 

build/int.cloudbuild.yaml

@@ -38,4 +38,4 @@ tags:
 - 'integration'
 substitutions:
   _DOCKER_IMAGE_DEVELOPER_TOOLS: 'cft/developer-tools'
-  _DOCKER_TAG_VERSION_DEVELOPER_TOOLS: '0.4.3'
+  _DOCKER_TAG_VERSION_DEVELOPER_TOOLS: '0'

build/lint.cloudbuild.yaml

@@ -21,4 +21,4 @@ tags:
 - 'lint'
 substitutions:
   _DOCKER_IMAGE_DEVELOPER_TOOLS: 'cft/developer-tools'
-  _DOCKER_TAG_VERSION_DEVELOPER_TOOLS: '0.4.3'
+  _DOCKER_TAG_VERSION_DEVELOPER_TOOLS: '0'

examples/dynamic-files/README.md

@@ -0,0 +1,74 @@
+# Automatic Labelling for folder projects
+
+This example demonstrates how to use the
+[root module][root-module] that will contain source
+code files generated from Terraform itself.
+
+## Usage
+
+To provision this example, populate `terraform.tfvars` with the [required variables](#inputs) and run the following commands within
+this directory:
+
+- `terraform init` to initialize the directory
+- `terraform plan` to generate the execution plan
+- `terraform apply` to apply the execution plan
+- `terraform destroy` to destroy the infrastructure
+
+<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
+## Inputs
+
+| Name | Description | Type | Default | Required |
+|------|-------------|:----:|:-----:|:-----:|
+| project\_id | The ID of the project to which resources will be applied. | string | n/a | yes |
+| region | The region in which resources will be applied. | string | n/a | yes |
+
+## Outputs
+
+| Name | Description |
+|------|-------------|
+| function\_name | The name of the function created |
+| random\_file\_string | The content of the terraform created file in the source directory. |
+| region | The region in which resources are applied. |
+
+<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
+
+## Requirements
+
+The following sections describe the requirements which must be met in
+order to invoke this module. The requirements of the
+[root module][root-module-requirements] and the
+[event-folder-log-entry submodule][event-folder-log-entry-submodule-requirements]
+must also be met.
+
+### Software Dependencies
+
+The following software dependencies must be installed on the system
+from which this module will be invoked:
+
+- [Terraform][terraform-site] v0.12
+
+### IAM Roles
+
+The Service Account which will be used to invoke this module must have
+the following IAM roles:
+
+- Logs Configuration Writer: `roles/logging.configWriter`
+- Pub/Sub Admin: `roles/pubsub.admin`
+- Service Account User: `roles/iam.serviceAccountUser`
+
+- Default AppSpot user: `roles/owner`
+- Your user: `roles/resourcemanager.projectCreator`
+
+### APIs
+
+The project against which this module will be invoked must have the
+following APIs enabled:
+
+- Cloud Pub/Sub API: `pubsub.googleapis.com`
+- Stackdriver Logging API: `logging.googleapis.com`
+
+[event-folder-log-entry-submodule-requirements]: ../../modules/event-folder-log-entry/README.md#requirements
+[event-folder-log-entry-submodule]: ../../modules/event-folder-log-entry
+[root-module-requirements]: ../../README.md#requirements
+[root-module]: ../..
+[terraform-site]: https://terraform.io/

examples/dynamic-files/function_source/index.js

@@ -0,0 +1,37 @@
+/**
+ * Copyright 2019 Google Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+const fs = require("fs");
+const path = require("path");
+const filePath = path.join(__dirname, "terraform_created_file.txt");
+
+/**
+ * Cloud function entrypoint
+ *
+ * @param {!Object} event Event payload and metadata.
+ * @param {!Function} callback Callback function to signal completion.
+ */
+exports.fileContent = (data, context, callback) => {
+  console.log("Received event");
+
+  fs.readFile(filePath, { encoding: "utf-8" }, function(err, data) {
+    if (!err) {
+      callback(null, data);
+    } else {
+      callback(err);
+    }
+  });
+};

examples/dynamic-files/function_source/package.json

@@ -0,0 +1,4 @@
+{
+  "name": "dynamic-files",
+  "version": "0.0.1"
+}

examples/dynamic-files/main.tf

@@ -0,0 +1,66 @@
+/**
+ * Copyright 2019 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+resource "random_pet" "main" {
+  length    = 2
+  separator = "-"
+}
+
+resource "google_storage_bucket" "trigger_bucket" {
+  name          = "${random_pet.main.id}-trigger"
+  force_destroy = true
+  location      = var.region
+  project       = var.project_id
+  storage_class = "REGIONAL"
+}
+
+resource "random_string" "random" {
+  length  = 16
+  special = false
+}
+
+resource "local_file" "file" {
+  content  = random_string.random.result
+  filename = "${path.module}/function_source/terraform_created_file.txt"
+}
+
+module "localhost_function" {
+  source = "../.."
+
+  description = "Returns back the random file content"
+  entry_point = "fileContent"
+
+  event_trigger = {
+    event_type = "google.storage.object.finalize"
+    resource   = google_storage_bucket.trigger_bucket.name
+  }
+
+  name             = random_pet.main.id
+  project_id       = var.project_id
+  region           = var.region
+  source_directory = "${path.module}/function_source"
+  runtime          = "nodejs8"
+
+  source_dependent_files = [local_file.file]
+}
+
+resource "null_resource" "wait_for_function" {
+  provisioner "local-exec" {
+    command = "sleep 60"
+  }
+
+  depends_on = [module.localhost_function]
+}