Implemented #48

README.md

@@ -5,18 +5,19 @@ A script to create, update and delete Zabbix hosts using Netbox device objects.
 
 
 ## Installation
-### Packages
-Make sure that you have a python environment with the following packages installed.
-```
-pynetbox
-pyzabbix
-```
 
 ### Cloning the repository
 ```
 git clone https://github.com/TheNetworkGuy/netbox-zabbix-sync.git
 ```
 
+### Packages
+Make sure that you have a python environment with the following packages installed. You can also use the requirements.txt file for installation with pip.
+```
+pynetbox
+pyzabbix
+```
+
 ### Config file
 First time user? Copy the config.py.example file to config.py. This file is used for modifying filters and setting variables such as custom field names.
 ```
@@ -52,6 +53,83 @@ You can make the hostID field hidden or read-only to prevent human intervention.
 
 This is optional and there is a use case for leaving it read-write in the UI to manually change the ID. For example to re-run a sync.
 
+## Config file
+
+### Hostgroup
+Setting the create_hostgroups variable to False requires manual hostgroup creation for devices in a new category.
+
+The format can be set with the hostgroup_format variable.
+
+Make sure that the Zabbix user has proper permissions to create hosts.
+The hostgroups are in a nested format. This means that proper permissions only need to be applied to the site name hostgroup and cascaded to any child hostgroups.
+
+#### layout
+The default hostgroup layout is "site/manufacturer/device_role".
+
+**Variables**
+
+You can change this behaviour with the hostgroup_format variable. The following values can be used:
+|  name | description  |
+| ------------ | ------------ |
+|dev_location|The device location name|
+|dev_role|The device role name|
+|manufacturer|Manufacturer name|
+|region|The region name of the device|
+|site|Site name|
+|site_group|Site group name|
+|tenant|Tenant name|
+|tenant_group|Tenant group name|
+
+
+You can specify the value like so, sperated by a "/":
+```
+hostgroup_format = "tenant/site/dev_location/dev_role"
+```
+**custom fields**
+
+You can also use the value of custom fields under the device object.
+
+This allows more freedom and even allows a full static mapping instead of a dynamic rendered hostgroup name.
+```
+hostgroup_format = "site/mycustomfieldname"
+```
+**Empty variables or hostgroups**
+
+Should the content of a variable be empty, then the hostgroup position is skipped.
+
+For example, consider the following scenario with 2 devices, both the same device type and site. One of them is linked to a tenant, the other one does not have a relationship with a tenant.
+- Device_role: PDU
+- Site: HQ-AMS
+```
+hostgroup_format = "site/tenant/device_role"
+```
+When running the script like above, the following hostgroup (HG) will be generated for both hosts:
+ - Device A with no relationship with a tenant: HQ-AMS/PDU
+ - Device B with a relationship to tenant "Fork Industries": HQ-AMS/Fork Industries/PDU
+
+The same logic applies to custom fields being used in the HG format:
+```
+hostgroup_format = "site/mycustomfieldname"
+```
+For device A with the value "ABC123" in the custom field "mycustomfieldname" -> HQ-AMS/ABC123
+For a device which does not have a value in the custom field "mycustomfieldname" -> HQ-AMS
+
+Should there be a scenario where a custom field does not have a value under a device, and the HG format only uses this single variable, then this will result in an error:
+```
+hostgroup_format = "mycustomfieldname"
+
+Netbox-Zabbix-sync - ERROR - ESXI1 has no reliable hostgroup. This is most likely due to the use of custom fields that are empty.
+```
+### Device status
+By setting a status on a Netbox device you determine how the host is added (or updated) in Zabbix. There are, by default, 3 options:
+* Delete the host from Zabbix (triggered by Netbox status "Decommissioning" and "Inventory")
+* Create the host in Zabbix but with a disabled status (Trigger by "Offline", "Planned", "Staged" and "Failed")
+* Create the host in Zabbix with an enabled status (For now only enabled with the "Active" status)
+
+You can modify this behaviour by changing the following list variables in the script:
+ - zabbix_device_removal
+ - zabbix_device_disable
+
 ### Template source
 You can either use a Netbox device type custom field or Netbox config context for the Zabbix template information.
 
@@ -106,83 +184,9 @@ python3 netbox_zabbix_sync.py
 ### Flags
 |  Flag | Option  |  Description |
 | ------------ | ------------ | ------------ |
-|  -c | cluster | For clustered devices: only add the primary node of a cluster and use the cluster name as hostname. |
-|  -H | hostgroup | Create non-existing hostgroups in Zabbix. Usefull for a first run to add all required hostgroups. |
-|  -l | layout | Set the hostgroup layout. Default is site/manufacturer/dev_role. Posible options (seperated with '/'): site, manufacturer, dev_role, tenant |
 |  -v | verbose | Log with debugging on. |
-|  -j | journal | Create journal entries in Netbox when a host gets added, modified or deleted in Zabbix |
-|  -p | proxy-power | Force a full proxy sync (includes deleting the proxy in Zabbix if not present in config context in Netbox) |
 
-#### Hostgroup
+## Config context
-In case of omitting the -H flag, manual hostgroup creation is required for devices in a new category.
-
-The format can be set with the -l flag. If not provided the default format will be:
-{Site name}/{Manufacturer name}/{Device role name}
-
-Make sure that the Zabbix user has proper permissions to create hosts.
-The hostgroups are in a nested format. This means that proper permissions only need to be applied to the site name hostgroup and cascaded to any child hostgroups.
-
-#### layout
-The default hostgroup layout is "site/manufacturer/device_role".
-
-**Variables**
-
-You can change this behaviour with the --layout flag. The following variables can be used:
-|  name | description  |
-| ------------ | ------------ |
-|tenant|Tenant name|
-|site|Site name|
-|manufacturer|Manufacturer name|
-|device_role|The device role name|
-
-You can specify the variables like so, sperated by a "/":
-```
-python3 netbox_zabbix_sync.py -l tenant/site/device_role
-```
-**custom fields**
-
-You can also use the value of custom fields under the device object.
-
-This allows more freedom and even allows a ful static mapping instead of a dynamic rendered hostgroup name.
-```
-python3 netbox_zabbix_sync.py -l site/mycustomfieldname
-```
-**Empty variables or hostgroups**
-
-Should the content of a variable be empty, then the hostgroup position is skipped.
-
-For example, consider the following scenario with 2 devices, both the same device type and site. One of them is linked to a tenant, the other one does not have a relationship with a tenant.
-- Device_role: PDU
-- Site: HQ-AMS
-```
-python3 netbox_zabbix_sync.py -l site/tenant/device_role
-```
-When running the script like above, the following hostgroup (HG) will be generated for both hosts:
- - Device A with no relationship with a tenant: HQ-AMS/PDU
- - Device B with a relationship to tenant "Fork Industries": HQ-AMS/Fork Industries/PDU
-
-The same logic applies to custom fields being used in the HG format:
-```
-python3 netbox_zabbix_sync.py -l site/mycustomfieldname
-```
-For device A with the value "ABC123" in the custom field "mycustomfieldname" -> HQ-AMS/ABC123
-For a device which does not have a value in the custom field "mycustomfieldname" -> HQ-AMS
-
-Should there be a scenario where a custom field does not have a value under a device, and the HG format only uses this signle variable, then this will result in an error:
-```
-python3 netbox_zabbix_sync.py -l mycustomfieldname
-
-Netbox-Zabbix-sync - ERROR - ESXI1 has no reliable hostgroup. This is most likely due to the use of custom fields that are empty.
-```
-### Device status
-By setting a status on a Netbox device you determine how the host is added (or updated) in Zabbix. There are, by default, 3 options:
-* Delete the host from Zabbix (triggered by Netbox status "Decommissioning" and "Inventory")
-* Create the host in Zabbix but with a disabled status (Trigger by "Offline", "Planned", "Staged" and "Failed")
-* Create the host in Zabbix with an enabled status (For now only enabled with the "Active" status)
-
-You can modify this behaviour by changing the following list variables in the script:
- - zabbix_device_removal
- - zabbix_device_disable
 
 ### Zabbix proxy
 You can set the proxy for a device using the 'proxy' key in config context.
@@ -193,7 +197,7 @@ You can set the proxy for a device using the 'proxy' key in config context.
     }
 }
 ```
-Because of the posible amount of destruction when setting up Netbox but forgetting the proxy command, the sync works a bit different. By default everything is synced except in a situation where the Zabbix host has a proxy configured but nothing is configured in Netbox. To force deletion and a full sync, use the -p flag.
+Because of the posible amount of destruction when setting up Netbox but forgetting the proxy command, the sync works a bit different. By default everything is synced except in a situation where the Zabbix host has a proxy configured but nothing is configured in Netbox. To force deletion and a full sync, set the full_proxy_sync variable in the config file.
 
 ### Set interface parameters within Netbox
 When adding a new device, you can set the interface type with custom context. By default, the following configuration is applied when no config context is provided:
@@ -256,4 +260,7 @@ To configure the interface parameters you'll need to use custom context. Custom
     }
 }
 ```
+
+I would recommend using macros for sensitive data such as community strings since the data in Netbox is plain-text.
+
 Note: Not all SNMP data is required for a working configuration. [The following parameters are allowed ](https://www.zabbix.com/documentation/current/manual/api/reference/hostinterface/object#details_tag "The following parameters are allowed ")but are not all required, depending on your environment.

config.py.example

@@ -11,10 +11,29 @@ templates_config_context_overrule = False
 template_cf = "zabbix_template"
 device_cf = "zabbix_hostid"
 
+# Enable clustering of devices with virtual chassis setup
+clustering = False
+
+# Enable hostgroup generation. Requires permissions in Zabbix
+create_hostgroups = True
+
+# Create journal entries
+create_journal = False
+
+# Set to true to enable removal of proxy's under hosts. Use with caution and make sure that you specified
+# all the required proxy's in the device config context before enabeling this option.
+# With this option disabled proxy's will only be added and modified for Zabbix hosts.
+full_proxy_sync = False
+
 # Netbox to Zabbix device state convertion
 zabbix_device_removal = ["Decommissioning", "Inventory"]
 zabbix_device_disable = ["Offline", "Planned", "Staged", "Failed"]
 
+# Hostgroup mapping
+# Available choices: dev_location, dev_role, manufacturer, region, site, site_group, tenant, tenant_group
+# You can also use CF (custom field) names under the device. The CF content will be used for the hostgroup generation.
+hostgroup_format = "site/manufacturer/dev_role"
+
 # Custom filter for device filtering. Variable must be present but can be left empty with no filtering.
 # A couple of examples are as follows:
 

netbox_zabbix_sync.py

@@ -7,7 +7,18 @@ import argparse
 from pynetbox import api
 from pyzabbix import ZabbixAPI, ZabbixAPIException
 try:
-    from config import *
+    from config import (
+        templates_config_context,
+        templates_config_context_overrule,
+        clustering, create_hostgroups,
+        create_journal, full_proxy_sync,
+        template_cf, device_cf,
+        zabbix_device_removal,
+        zabbix_device_disable,
+        hostgroup_format,
+        nb_device_filter
+    )
+    
 except ModuleNotFoundError:
     print(f"Configuration file config.py not found in main directory."
            "Please create the file or rename the config.py.example file to config.py.")
@@ -52,20 +63,19 @@ def main(arguments):
     # Set Netbox API
     netbox = api(netbox_host, token=netbox_token, threading=True)
     # Check if the provided Hostgroup layout is valid
-    if(arguments.layout):
+    hg_objects = hostgroup_format.split("/")
-        hg_objects = arguments.layout.split("/")
+    allowed_objects = ["dev_location", "dev_role", "manufacturer", "region",
-        allowed_objects = ["dev_location", "dev_role", "manufacturer", "region",
+                        "site", "site_group", "tenant", "tenant_group"]
-                           "site", "site_group", "tenant", "tenant_group"]
+    # Create API call to get all custom fields which are on the device objects
-        # Create API call to get all custom fields which are on the device objects
+    device_cfs = netbox.extras.custom_fields.filter(type="text", content_type_id=23)
-        device_cfs = netbox.extras.custom_fields.filter(type="text", content_type_id=23)
+    for cf in device_cfs:
-        for cf in device_cfs:
+        allowed_objects.append(cf.name)
-            allowed_objects.append(cf.name)
+    for object in hg_objects:
-        for object in hg_objects:
+        if(object not in allowed_objects):
-            if(object not in allowed_objects):
+            e = (f"Hostgroup item {object} is not valid. Make sure you"
-                e = (f"Hostgroup item {object} is not valid. Make sure you"
+                    " use valid items and seperate them with '/'.")
-                     " use valid items and seperate them with '/'.")
+            logger.error(e)
-                logger.error(e)
+            raise HostgroupError(e)
-                raise HostgroupError(e)
     # Set Zabbix API
     try:
         zabbix = ZabbixAPI(zabbix_host)
@@ -83,12 +93,12 @@ def main(arguments):
     for nb_device in netbox_devices:
         try:
             device = NetworkDevice(nb_device, zabbix, netbox_journals,
-                                   arguments.journal)
+                                   create_journal)
-            device.set_hostgroup(arguments.layout)
+            device.set_hostgroup(hostgroup_format)
             device.set_template(templates_config_context, templates_config_context_overrule)
             # Checks if device is part of cluster.
-            # Requires the cluster argument.
+            # Requires clustering variable
-            if(device.isCluster() and arguments.cluster):
+            if(device.isCluster() and clustering):
                 # Check if device is master or slave
                 if(device.promoteMasterDevice()):
                     e = (f"Device {device.name} is "
@@ -117,9 +127,9 @@ def main(arguments):
                 continue
             elif(device.status in zabbix_device_disable):
                 device.zabbix_state = 1
-            # Add hostgroup is flag is true
+            # Add hostgroup is variable is True
             # and Hostgroup is not present in Zabbix
-            if(arguments.hostgroups):
+            if(create_hostgroups):
                 for group in zabbix_groups:
                     # If hostgroup is already present in Zabbix
                     if(group["name"] == device.hostgroup):
@@ -131,7 +141,7 @@ def main(arguments):
             # Device is already present in Zabbix
             if(device.zabbix_id):
                 device.ConsistencyCheck(zabbix_groups, zabbix_templates,
-                                        zabbix_proxys, arguments.proxy_power)
+                                        zabbix_proxys, full_proxy_sync)
             # Add device to Zabbix
             else:
                 device.createInZabbix(zabbix_groups, zabbix_templates,
@@ -612,7 +622,7 @@ class NetworkDevice():
         else:
             if(not host["proxy_hostid"] == "0"):
                 if(proxy_power):
-                    # If the -p flag has been issued,
+                    # Variable full_proxy_sync has been enabled
                     # delete the proxy link in Zabbix
                     self.updateZabbixHost(proxy_hostid=self.zbxproxy)
                 else:
@@ -820,29 +830,10 @@ class ZabbixInterface():
 
 
 if(__name__ == "__main__"):
-    # Arguments parsing
     parser = argparse.ArgumentParser(
         description='A script to sync Zabbix with Netbox device data.'
     )
     parser.add_argument("-v", "--verbose", help="Turn on debugging.",
                         action="store_true")
-    parser.add_argument("-c", "--cluster", action="store_true",
-                        help=("Only add the primary node of a cluster "
-                              "to Zabbix. Usefull when a shared virtual IP is "
-                              "used for the control plane."))
-    parser.add_argument("-H", "--hostgroups",
-                        help="Create Zabbix hostgroups if not present",
-                        action="store_true")
-    parser.add_argument("-l", "--layout", type=str,
-                        help="Defines the hostgroup layout",
-                        default='site/manufacturer/dev_role')
-    parser.add_argument("-p", "--proxy_power", action="store_true",
-                        help=("USE WITH CAUTION. If there is a proxy "
-                              "configured in Zabbix but not in Netbox, sync "
-                              "the device and remove the host - proxy "
-                              "link in Zabbix."))
-    parser.add_argument("-j", "--journal", action="store_true",
-                        help="Create journal entries in Netbox at write actions")
     args = parser.parse_args()
-
     main(args)