Merge pull request #22 from netboxlabs/features-1.1.0

Features 1.1.0

Author: netboxlabs-nvp

.gitignore

@@ -173,4 +173,6 @@ inventory
 tests/
 
 app_config.yml
-
+netbox_setup_custom_fields.yml
+netbox_importer.py*
+netbox_setup_objects.yml

conf.d/netbox_setup_objects.yml-sample

@@ -0,0 +1,33 @@
+proxmox_api_config:
+  api_host: proxmox-ip-or-hostname
+  api_port: 8006
+  api_user: proxmox_api_user
+  api_token_id: name_of_proxmox_api_token
+  api_token_secret: proxmox_api_secret_token
+  verify_ssl: false
+netbox_api_config:
+  api_proto: http # or https
+  api_host: name or ip of NetBox host
+  api_port: 8000
+  api_token: netbox_api_secret_token
+  verify_ssl: false # or true, up to you
+proxmox:
+  cluster_name: proxmox-ve
+netbox:
+  cluster_role: Proxmox
+  vm_role: "Proxmox VM"
+automation_type: choices are ansible_automation or flask_application
+ansible_automation:
+  host: name or ip of AWX/Tower/AAP host
+  http_proto: http or https
+  http_port: 80 or whatever
+  ssl_verify: false # or true
+  username: awx_user # should have permissions to view both projects and templates
+  password: awx_password
+  project_name: netbox-proxmox-ee-test1 # or whatever you named your project
+flask_application:
+  host: name or ip of where Flask application is running
+  http_proto: http or https
+  http_port: 9000 or whatever
+  ssl_verify: false # or true
+  netbox_webhook_name: "netbox-proxmox-webhook"

docs/netbox-customization.md

@@ -10,8 +10,56 @@ In the NetBox UI, you will need to create the following custom field choices
 2. `proxmox-vm-templates` will be used to correlate a Proxmox VM template with the Proxmox VM that you will provision
 3. `proxmox-vm-storage` will be used to correlate an underlying Proxmox VM storage volume with the Proxmox VM you will provision
 
-In the NetBox UI, navigate to Customization > Custom Field Choices
+#### Automated NetBox Objects and Custom Fields Creation
 
+If you'd prefer to manually create the webhook and event rules in NetBox, you can skip to the next section.  Otherwise, proceed with the following to automate the creation of the webhook and event rules in NetBox.
+
+`netbox-proxmox-automation` version 1.1.0 and newer ships with a convenience script, `netbox_setup_objects_and_custom_fields.py`, that when used alongside a configuration file of your choice, will greatly simplify this process.  In the case of AWX/Tower/AAP, `netbox_setup_objects_and_custom_fields.py` will query your AWX/Tower/AAP instance for project and template(s) information; this information will then be used to create the corresponding webhooks and event rules in NetBox.
+
+There exists a sample configuration file called `netbox_setup_objects.yml-sample` under the conf.d directory of this git repository.  Copy this file to a location of your choice, and season it to taste.  In the end you should have a configuration that looks something like this.
+
+```
+proxmox_api_config:
+  api_host: proxmox-ip-or-hostname
+  api_port: 8006
+  api_user: proxmox_api_user
+  api_token_id: name_of_proxmox_api_token
+  api_token_secret: proxmox_api_secret_token
+  verify_ssl: false
+netbox_api_config:
+  api_proto: http # or https
+  api_host: name or ip of NetBox host
+  api_port: 8000
+  api_token: netbox_api_secret_token
+  verify_ssl: false # or true, up to you
+proxmox:
+  cluster_name: proxmox-ve
+netbox:
+  cluster_role: Proxmox
+  vm_role: "Proxmox VM"
+```
+
+Usage:
+
+```
+shell$ cd setup
+
+shell$ pwd
+/some/path/netbox-proxmox-automation/setup
+
+shell$ python3 -m venv venv
+
+shell$ source venv/bin/activate
+
+(venv) shell$ pip install -r requirements.txt
+
+(venv) shell$ ./netbox_setup_objects_and_custom_fields.py --config /path/to/your/configuration.yml
+```
+
+Then verify that everything has been created.  You can skip the rest of this document if so.
+
+
+If *not* using automation, in the NetBox UI, navigate to Customization > Custom Field Choices
 
 #### proxmox-node
 

docs/netbox-event-rules-and-webhooks-awx-aap.md

@@ -24,6 +24,63 @@ Regardless of whether you are using a Flask (or other) application for Proxmox a
 
 As noted earlier, AWX/Tower/AAP will perform Proxmox automation through separate (job) templates.  This section walks you through how (NetBox) webhooks and (NetBox) event rules are handled by AWX.
 
+#### Automated Webhook and Event Rules Configuration
+
+If you'd prefer to manually create the webhook and event rules in NetBox, you can skip to the next section.  Otherwise, proceed with the following to automate the creation of the webhook and event rules in NetBox.
+
+`netbox-proxmox-automation` version 1.1.0 and newer ships with a convenience script, `netbox_setup_webhook_and_event_rules.py`, that when used alongside a configuration file of your choice, will greatly simplify this process.  In the case of AWX/Tower/AAP, `netbox_setup_webhook_and_event_rules.py` will query your AWX/Tower/AAP instance for project and template(s) information; this information will then be used to create the corresponding webhooks and event rules in NetBox.
+
+There exists a sample configuration file called `netbox_setup_objects.yml-sample` under the conf.d directory of this git repository.  Copy this file to a location of your choice, and season it to taste.  In the end you should have a configuration that looks something like this.
+
+```
+proxmox_api_config:
+  api_host: proxmox-ip-or-hostname
+  api_port: 8006
+  api_user: proxmox_api_user
+  api_token_id: name_of_proxmox_api_token
+  api_token_secret: proxmox_api_secret_token
+  verify_ssl: false
+netbox_api_config:
+  api_proto: http # or https
+  api_host: name or ip of NetBox host
+  api_port: 8000
+  api_token: netbox_api_secret_token
+  verify_ssl: false # or true, up to you
+proxmox:
+  cluster_name: proxmox-ve
+netbox:
+  cluster_role: Proxmox
+  vm_role: "Proxmox VM"
+automation_type: ansible_automation
+ansible_automation:
+  host: name or ip of AWX/Tower/AAP host
+  http_proto: http or https
+  http_port: 80 or whatever
+  ssl_verify: false # or true
+  username: awx_user # should have permissions to view both projects and templates
+  password: awx_password
+  project_name: netbox-proxmox-ee-test1 # or whatever you named your project
+```
+
+Usage:
+
+```
+shell$ cd setup
+
+shell$ pwd
+/some/path/netbox-proxmox-automation/setup
+
+shell$ python3 -m venv venv
+
+shell$ source venv/bin/activate
+
+(venv) shell$ pip install -r requirements.txt
+
+(venv) shell$ ./netbox_setup_webhook_and_event_rules.py --config /path/to/your/configuration.yml
+```
+
+Then verify that everything has been created.  You can skip the rest of this document if so.
+
 #### AWX/Tower/AAP Webhook
 
 To use NetBox webhooks with AWX, each NetBox webhook for Proxmox VM management will point at a separate AWX (job) template.  In AWX, each (job) template has a unique ID.  When we execute a webhook in NetBox, in this case we're using AWX, the (NetBox) webhook will in turn point at the (job) template ID in AWX -- and tell AWX to launch the template, i.e. to run the automation.

docs/netbox-event-rules-and-webhooks-flask.md

@@ -24,6 +24,61 @@ Regardless of whether you are using a Flask (or other) application for Proxmox a
 
 As noted [here](#initial-configuration-flask-application-python), you will need to have a running Flask application *before* you can start handling events (i.e. object changes) inside of NetBox.
 
+#### Automated Webhook and Event Rules Configuration
+
+If you'd prefer to manually create the webhook and event rules in NetBox, you can skip to the next section.  Otherwise, proceed with the following to automate the creation of the webhook and event rules in NetBox.
+
+`netbox-proxmox-automation` version 1.1.0 and newer ships with a convenience script, `netbox_setup_webhook_and_event_rules.py`, that when used alongside a configuration file of your choice, will greatly simplify this process.
+
+There exists a sample configuration file called `netbox_setup_objects.yml-sample` under the conf.d directory of this git repository.  Copy this file to a location of your choice, and season it to taste.  In the end you should have a configuration that looks something like this.
+
+```
+proxmox_api_config:
+  api_host: proxmox-ip-or-hostname
+  api_port: 8006
+  api_user: proxmox_api_user
+  api_token_id: name_of_proxmox_api_token
+  api_token_secret: proxmox_api_secret_token
+  verify_ssl: false
+netbox_api_config:
+  api_proto: http # or https
+  api_host: name or ip of NetBox host
+  api_port: 8000
+  api_token: netbox_api_secret_token
+  verify_ssl: false # or true, up to you
+proxmox:
+  cluster_name: proxmox-ve
+netbox:
+  cluster_role: Proxmox
+  vm_role: "Proxmox VM"
+automation_type: flask_application
+flask_application:
+  host: name or ip of where Flask application is running
+  http_proto: http or https
+  http_port: 9000 or whatever
+  ssl_verify: false # or true
+  netbox_webhook_name: "netbox-proxmox-webhook"
+```
+
+Usage:
+
+```
+shell$ cd setup
+
+shell$ pwd
+/some/path/netbox-proxmox-automation/setup
+
+shell$ python3 -m venv venv
+
+shell$ source venv/bin/activate
+
+(venv) shell$ pip install -r requirements.txt
+
+(venv) shell$ ./netbox_setup_webhook_and_event_rules.py --config /path/to/your/configuration.yml
+```
+
+Then verify that everything has been created.  You can skip the rest of this document if so.
+
 #### Flask Application: Webhook
 
 `example-netbox-webhook-flask-app` implements a catch-all for virtual machine events that happen in NetBox.  Events will call the webhook, and in turn the webhook will dispatch Proxmox VM changes via the Proxmox API.

example-netbox-webhook-flask-app/app.py

@@ -2,13 +2,17 @@
 import json
 import yaml
 
+from datetime import datetime
+
 # adapted from: https://majornetwork.net/2019/10/webhook-listener-for-netbox/
 
 from helpers.netbox_proxmox import NetBoxProxmoxHelper
 
-from flask import Flask, request
+from flask import Flask, request, jsonify
 from flask_restx import Api, Resource, fields
 
+VERSION = '1.1.0'
+
 app_config_file = 'app_config.yml'
 
 with open(app_config_file) as yaml_cfg:
@@ -50,7 +54,33 @@
     'request_id': fields.String,
 })
 
+# For session logging, c/o sol1
+session = {
+  'name': "example-netbox-webhook-flask-app",
+  'version': VERSION,
+  'version_lastrun': VERSION,
+  'server_start': "",
+  'status': {
+    'requests': 0,
+    'last_called': ""
+  },
+}
+
+
+@ns.route("/status/", methods=['GET'])
+class WebhookListener(Resource):
+    @ns.expect(webhook_request)
+
+    def get(self):
+        _session = session.copy()
+        _session['version_lastrun'] = VERSION
+        _session['status']['requests'] += 1
+        _session['status']['last_called'] = datetime.now()
+        logger.info(f"{request.full_path}, {request.remote_addr}, Status request with data {request.get_data()}")
+        return jsonify(_session)
+
 
+# For handling event rules
 @ns.route("/")
 class WebhookListener(Resource):
     @ns.expect(webhook_request)

legacy/README.TXT

@@ -0,0 +1 @@
+Simply put, don't use this stuff.  It's just parked here and will be reimagined at some point in time.