117 Commits
1.0 ... 3.0

Author SHA1 Message Date
Twan K.
acab7dd6d2 Merge pull request #80 from TheNetworkGuy/virtual_machines
Virtual machines
2024-11-18 13:38:35 +01:00
TheNetworkGuy
2177234d7f Fixed some documentation 2024-11-18 13:27:24 +01:00
TheNetworkGuy
3f4d173ac0 Markdown fixed on custom field in README 2024-11-18 12:59:50 +01:00
TheNetworkGuy
0996059c5f Added better documentation for VMs nd fixed typo 2024-11-18 12:58:57 +01:00
TheNetworkGuy
0155c29fcc Fixed Pylint too-many-public-methods error 2024-11-15 14:08:04 +01:00
TheNetworkGuy
5d4ff9c5ed Fixed #79 2024-11-15 14:03:42 +01:00
TheNetworkGuy
204937b784 Fixed variable name typo 2024-11-13 20:32:47 +01:00
TheNetworkGuy
e0827ac428 Fixed long line linter error 2024-11-13 20:00:50 +01:00
TheNetworkGuy
09a6906a63 Added VM filtering 2024-11-13 19:56:09 +01:00
TheNetworkGuy
30545ec0f3 Added hostname filtering based on Cyrillic characters 2024-11-13 19:39:24 +01:00
TheNetworkGuy
56c19d97de Added basic error message when Netbox details are invalid. Fixed logging message for when template context keys are not present. 2024-11-06 15:57:11 +01:00
TheNetworkGuy
ffc2aa1947 Fixed location hostgroup bug 2024-10-31 20:03:09 +01:00
TheNetworkGuy
9417908994 Swapped hostgroup and template calculation since template lookup is a read-only operation and hostgroup lookup could be a read-write operation. 2024-10-31 15:51:33 +01:00
TheNetworkGuy
06f97b132a Changed some logging messages and removed import logging statement from troubleshooting 2024-10-30 21:25:58 +01:00
TheNetworkGuy
20096a215b Added HG and template checks for devices and VM's. Temp disabled inventory mapping for VMs 2024-10-30 21:10:22 +01:00
TheNetworkGuy
f1da1cfb50 Fixed logging formatting for submodules 2024-10-30 20:57:20 +01:00
TheNetworkGuy
5093823287 Added some logging and fixed role assignment for VM's 2024-10-30 20:52:28 +01:00
TheNetworkGuy
c1504987f1 Fixed bug for Tenant hostgroup generation 2024-10-30 18:21:42 +01:00
TheNetworkGuy
d598a9739a Fixed whitespace 2024-10-30 13:51:30 +01:00
TheNetworkGuy
7bf72de0f9 Fixed bug where a single host exception would stop the sync. 2024-10-30 13:50:20 +01:00
TheNetworkGuy
66f24e6891 Fixed several bugs, set default interface for VM to agent, fixed several linter errors. 2024-10-30 12:23:15 +01:00
TheNetworkGuy
bff34a8e38 Renamed VM module for python naming convention. Fixed 2 VM hostgroup bugs. Fixed some whitespace in the VM module. 2024-10-30 10:06:29 +01:00
TheNetworkGuy
886ef2a172 Replaced old device role field with new one 2024-10-25 18:56:58 +02:00
TheNetworkGuy
9c07d7dbc4 Updated Readme with VM info 2024-10-25 18:54:17 +02:00
TheNetworkGuy
9f29d2b27b Added basic VM support 2024-10-25 18:46:20 +02:00
TheNetworkGuy
e827953d8d Fixed extra space in config.py.example 2024-10-25 16:50:56 +02:00
TheNetworkGuy
053028b283 Splitted hostgroup generation logic into its seperate module. Changed hostgroup "dev_role" to "role" for VM role prepration. Started work on basic VM class. 2024-10-25 16:02:08 +02:00
TheNetworkGuy
2e867d1129 Added .venv to gitignore for developing. 2024-10-25 15:53:33 +02:00
TheNetworkGuy
a0ea21d731 Fixed bug in which some instances a nested hostgroup was not created 2024-07-25 16:23:59 +02:00
TheNetworkGuy
70a5c3e384 Implemented #73 2024-07-25 15:47:09 +02:00
TheNetworkGuy
91796395ef Fixed readme documentation with inventory settings 2024-07-24 15:02:31 +02:00
TheNetworkGuy
610a73c061 Merge branch 'main' of https://github.com/TheNetworkGuy/netbox-zabbix-sync 2024-07-24 14:31:33 +02:00
TheNetworkGuy
4de022496e Fixed invalid mapper log message even when not using the inventory mapping function. 2024-07-24 14:31:29 +02:00
Twan K.
0603d8c244 Merge pull request #71 from q1x/allow-proxy-null-value
Allow proxy null value
2024-07-24 14:26:21 +02:00
TheNetworkGuy
2b92f8da9b Fixed #72 2024-07-24 14:25:22 +02:00
TheNetworkGuy
d1ec1114ac Fixes #70 2024-07-24 13:14:12 +02:00
Raymond Kuiper
acad07eed4 Merge pull request #2 from q1x/main
Allow for NetBox Config Context to have a null value for proxy.
2024-07-24 11:07:29 +02:00
Raymond Kuiper
da4fec6bf1 Removed trailing whitespace 2024-07-24 11:06:50 +02:00
Raymond Kuiper
07049ea6d8 Adjustment from linting suggestion 2024-07-24 11:05:35 +02:00
Raymond Kuiper
2094799a51 Allow for NetBox Config Context to have a null value for proxy. 2024-07-24 10:58:22 +02:00
Twan K.
c0c52f973e Merge pull request #69 from doteater/patch-1
Update README.md
2024-07-24 09:59:41 +02:00
doteater
39b63aa420 Update README.md
reference to "latest" container label updated to "main". Also case issue with repo name.
2024-07-10 14:26:00 -07:00
Twan K
017b5623f5 Merge pull request #68 from q1x/main
corrected minor typo
2024-07-04 14:42:41 +02:00
Raymond Kuiper
9be09bca10 corrected minor typo 2024-07-04 13:26:58 +02:00
TheNetworkGuy
23997f9423 Properly styled code as suggested in #61. Also implemented #63 2024-06-12 19:02:44 +02:00
TheNetworkGuy
e8a733cbd0 Merge branch 'main' of https://github.com/TheNetworkGuy/netbox-zabbix-sync 2024-06-12 15:29:37 +02:00
TheNetworkGuy
be76386584 Removed the init file since it interfeared with the linter. 2024-06-12 15:29:33 +02:00
Twan K
b5a01e09e8 Merge pull request #62 from TheNetworkGuy/zabbix_utils
Zabbix 7 support and switching over to zabbix_utils.
2024-06-12 15:01:50 +02:00
TheNetworkGuy
ecec3ee46e Fixed authentication variable problems 2024-06-12 14:57:35 +02:00
TheNetworkGuy
7099df93d1 Updated documentation and requirements 2024-06-12 14:03:37 +02:00
TheNetworkGuy
d1e864c75b Fixed hostgroup generation issues, added proxygroup support. 2024-06-12 13:42:21 +02:00
TheNetworkGuy
6f044cb228 Added logic for proxy groups. Only thing left is consistency check 2024-06-11 16:13:45 +02:00
TheNetworkGuy
2e7890784b Changed flow in sync script, started work for Proxy group support and zabbix_utils 2024-06-11 15:11:55 +02:00
TheNetworkGuy
c695353fce Splitted modules up into seperate files. 2024-06-10 20:20:38 +02:00
TheNetworkGuy
e0b473a6d4 Optimized code and also fixed Linter error. 2024-06-05 10:09:26 +02:00
Twan K
8e9594172b Merge pull request #59 from Kage1/main
Update README.py to reflect Netbox code change
2024-06-05 09:16:42 +02:00
Kage1
8a749e63cf Merge pull request #1 from Kage1/Kage1-patch-1
Update README.md
2024-05-30 13:52:40 -05:00
Kage1
ddc65a6d58 Update README.md
Per https://github.com/netbox-community/netbox/releases/v3.5.0 obj. has been updated to object.

Revised README to reflect this change.
2024-05-30 13:50:37 -05:00
TheNetworkGuy
58d894832e Merge branch 'main' of https://github.com/TheNetworkGuy/netbox-zabbix-sync 2024-05-22 14:34:07 +02:00
TheNetworkGuy
b9713792d7 fixed bug with device removal state 2024-05-22 14:34:03 +02:00
Twan K
45192531f9 Merge pull request #58 from TheNetworkGuy/hostname_processing
Added functionallity to allow device names with specials characters
2024-05-22 14:09:23 +02:00
TheNetworkGuy
72fde13ef4 Added functionallity to allow device names with specials characters such as umlauts. 2024-05-22 14:04:42 +02:00
TheNetworkGuy
78b9d5ae8b Fixed trailing white space error. 2024-05-22 10:38:53 +02:00
Twan K
60140b4b74 Merge pull request #56 from q1x/inventory-zero-values
Allow for source inventory fields to have a value of zero
2024-05-22 10:36:11 +02:00
TheNetworkGuy
0b9b8a4898 Fixed container path 2024-05-22 09:40:28 +02:00
TheNetworkGuy
441d7e7e95 Temporary disabled too-many-arguments error for Pylint. 2024-05-22 09:35:33 +02:00
TheNetworkGuy
c185b7364d Fixes #57. 2024-05-22 09:24:17 +02:00
Twan K
e56451f5e1 Update README.md
Fixed Docker package link
2024-05-21 15:38:10 +02:00
Twan K
dee6a079a5 Update README.md 2024-05-21 15:23:20 +02:00
Twan K
2b62caca85 Update README.md
Updated to force pipeline container build
2024-05-21 15:06:41 +02:00
Raymond Kuiper
e9143eb24c Update netbox_zabbix_sync.py
Minor bug fix for float or int zero values in inventory fields
2024-04-12 10:38:43 +02:00
Twan K
4eed151e22 Merge pull request #55 from q1x/main
Code cleanup and automated GitHub pylint workflow
2024-04-10 22:05:34 +02:00
Raymond Kuiper
3e638c6f78 Update netbox_zabbix_sync.py
Minor bug fix for empty Netbox to zabbix inventory field mappings.
2024-04-10 14:57:08 +02:00
Raymond Kuiper
634f4b77d5 tweaked exception handling 2024-03-28 09:51:08 +01:00
Raymond Kuiper
c006e7feb5 Let's make pylint happy :) 2024-03-27 20:35:32 +01:00
Raymond Kuiper
091c9746c0 Fixed proxy issue, rewrite of inventory logic (eval was ugly) 2024-03-27 20:33:02 +01:00
Raymond Kuiper
364d376f55 corrected even more linting errors 2024-03-27 16:33:06 +01:00
Raymond Kuiper
ab2a341fa7 Corrected more linting errors 2024-03-27 16:26:15 +01:00
Raymond Kuiper
fbb9eeb48c Corrected linting errors 2024-03-27 16:24:26 +01:00
Raymond Kuiper
5b08d27a5e Added support for syncing Zabbix Inventory, this is also a fix for https://github.com/TheNetworkGuy/netbox-zabbix-sync/issues/44 2024-03-27 15:37:50 +01:00
Raymond Kuiper
583d845c40 revert because of file formatting issue 2024-03-27 15:22:43 +01:00
Raymond Kuiper
27a4a5c6eb Corrected more pylint errors 2024-03-25 11:57:46 +01:00
Raymond Kuiper
537710a4b9 Corrected pylint errors 2024-03-25 11:56:17 +01:00
Raymond Kuiper
5defc1a25e Corrected another MarkDown error 2024-03-25 11:52:10 +01:00
Raymond Kuiper
d6973dc32d Corrected MarkDown error 2024-03-25 11:51:12 +01:00
Raymond Kuiper
71f604a6f6 Added functionality to build full region and site_group paths to be used in hostgroup names. 2024-03-25 11:49:41 +01:00
Raymond Kuiper
b94a0df02d Merge pull request #1 from q1x/dockertest
Added Dockerfile and workflow to build images
2024-03-13 14:29:41 +01:00
Raymond Kuiper
3079a88de8 better docker logs example. 2024-03-13 14:00:03 +01:00
Raymond Kuiper
4aa8b6d2fb updated README.md with Docker instructions 2024-03-13 13:55:46 +01:00
Raymond Kuiper
e82631c89d modified tags 2024-03-13 13:16:22 +01:00
Raymond Kuiper
18d29c98d3 updated tags 2024-03-13 11:58:40 +01:00
Raymond Kuiper
661ce88287 updated versions 2024-03-13 11:50:43 +01:00
Raymond Kuiper
4b7f3ec0b9 try a different way of publishing 2024-03-13 11:47:45 +01:00
Raymond Kuiper
3a39c314be removed on: push from pylint 2024-03-13 11:42:28 +01:00
Raymond Kuiper
bf325c6839 testing workflow 2024-03-13 11:38:07 +01:00
Raymond Kuiper
5922d3e8ae allow call from another workflow 2024-03-11 11:06:31 +01:00
Raymond Kuiper
dcd84e836b Chained in quality check 2024-03-11 11:03:37 +01:00
Raymond Kuiper
33cf3e5358 changed back to checkout@v4 and commited Dockerfile 2024-03-08 22:53:34 +01:00
Raymond Kuiper
7c988f9ff8 changed over to checkout@v3 2024-03-08 22:51:53 +01:00
Raymond Kuiper
d46b749af0 corrected typos 2024-03-08 22:48:57 +01:00
Raymond Kuiper
e05c35a3ea added container building workflow 2024-03-08 22:44:25 +01:00
Raymond Kuiper
142aae75e0 removed directory 2024-03-08 14:56:30 +01:00
Raymond Kuiper
c538c51b7b minor README.md update 2024-03-08 14:54:48 +01:00
Raymond Kuiper
0d7c581ee2 fixed undefined-variable 2024-03-08 14:29:12 +01:00
Raymond Kuiper
c684ac4a9d Futher cleanup 2024-03-08 14:23:47 +01:00
Raymond Kuiper
2fcd21a723 code cleanup 2024-03-08 14:00:15 +01:00
Raymond Kuiper
23bef6b549 disable pylint module name checks 2024-03-08 13:52:08 +01:00
Raymond Kuiper
0d02e096e9 Disable pylint name checking 2024-03-08 13:49:18 +01:00
Raymond Kuiper
3c7079117a changed versions for workflow (again) 2024-03-08 13:40:31 +01:00
Raymond Kuiper
89d5f22064 changed versions for workflow 2024-03-08 13:36:42 +01:00
Raymond Kuiper
15d40873b0 rename actions to workflows 2024-03-08 13:32:29 +01:00
Raymond Kuiper
de8143e89f Cleaning up code 2024-03-08 13:30:21 +01:00
Twan K
173fdbf19f Merge pull request #54 from q1x/zabbix-api-token
Support for Zabbix api token
2024-03-06 17:37:00 +01:00
Twan K
d55bb4053b Merge pull request #53 from q1x/zabbix-7.0.0-compatibility
Zabbix 7.0.0 compatibility
2024-03-06 17:35:27 +01:00
Raymond Kuiper
c8e42b366f Added support for Zabbix API tokens and updated README.md 2024-03-01 08:51:07 +01:00
Raymond Kuiper
1f4a81e2e4 Implemented workaround for Zabbix 7.0.x API changes. 2024-03-01 03:33:31 +01:00
Raymond Kuiper
8aba95525b Update in Zabbix proxy logic
- Fixed var typo (`proxys` vs `proxies`)
- Workaround for Zabbix 7.0.x API changes in `proxy.get()`: `host` was replaced with `name`.
2024-02-23 15:53:48 +01:00
15 changed files with 1645 additions and 787 deletions

46
.github/workflows/publish-image.yml vendored Normal file
View File

@@ -0,0 +1,46 @@
name: Publish Docker image to GHCR on a new version
on:
push:
branches:
- main
- dockertest
# tags:
# - [0-9]+.*
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
test_quality:
uses: ./.github/workflows/quality.yml
build_and_publish:
runs-on: ubuntu-latest
steps:
- name: Checkout sources
uses: actions/checkout@v4
- name: Log in to the container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GHCR_PAT }}
- name: Extract metadata (tags, labels)
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=semver,pattern={{ version }}
type=ref,event=branch
type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', 'master') }}
type=sha
- name: Build and push Docker image
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}

26
.github/workflows/quality.yml vendored Normal file
View File

@@ -0,0 +1,26 @@
---
name: Pylint Quality control
on:
workflow_call
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.11","3.12"]
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pylint
pip install -r requirements.txt
- name: Analysing the code with pylint
run: |
pylint --module-naming-style=any $(git ls-files '*.py')

1
.gitignore vendored
View File

@@ -1,4 +1,5 @@
*.log
.venv
config.py
# Byte-compiled / optimized / DLL files
__pycache__/

9
Dockerfile Normal file
View File

@@ -0,0 +1,9 @@
# syntax=docker/dockerfile:1
FROM python:3.12-alpine
RUN mkdir -p /opt/netbox-zabbix
COPY . /opt/netbox-zabbix
WORKDIR /opt/netbox-zabbix
RUN if ! [ -f ./config.py ]; then cp ./config.py.example ./config.py; fi
RUN pip install -r ./requirements.txt
ENTRYPOINT ["python"]
CMD ["/opt/netbox-zabbix/netbox_zabbix_sync.py", "-v"]

164
README.md
View File

@@ -4,7 +4,33 @@
A script to create, update and delete Zabbix hosts using Netbox device objects.
## Installation
## Installation via Docker
To pull the latest stable version to your local cache, use the following docker pull command:
```
docker pull ghcr.io/thenetworkguy/netbox-zabbix-sync:main
```
Make sure to specify the needed environment variables for the script to work (see [here](#set-environment-variables))
on the command line or use an [env file](https://docs.docker.com/reference/cli/docker/container/run/#env).
```
docker run -d -t -i -e ZABBIX_HOST='https://zabbix.local' \
-e ZABBIX_TOKEN='othersecrettoken' \
-e NETBOX_HOST='https://netbox.local' \
-e NETBOX_TOKEN='secrettoken' \
--name netbox-zabbix-sync ghcr.io/thenetworkguy/netbox-zabbix-sync:main
```
This should run a one-time sync, you can check the sync with `docker logs netbox-zabbix-sync`.
The image uses the default `config.py` for it's configuration, you can use a volume mount in the docker run command
to override with your own config file if needed (see [config file](#config-file)):
```
docker run -d -t -i -v $(pwd)/config.py:/opt/netbox-zabbix/config.py ...
```
## Installation from Source
### Cloning the repository
```
@@ -12,20 +38,20 @@ git clone https://github.com/TheNetworkGuy/netbox-zabbix-sync.git
```
### Packages
Make sure that you have a python environment with the following packages installed. You can also use the requirements.txt file for installation with pip.
Make sure that you have a python environment with the following packages installed. You can also use the `requirements.txt` file for installation with pip.
```
pynetbox
pyzabbix
```
### Config file
First time user? Copy the config.py.example file to config.py. This file is used for modifying filters and setting variables such as custom field names.
First time user? Copy the `config.py.example` file to `config.py`. This file is used for modifying filters and setting variables such as custom field names.
```
cp config.py.example config.py
```
### Set environment variables
Set the following environment variables
Set the following environment variables:
```
ZABBIX_HOST="https://zabbix.local"
ZABBIX_USER="username"
@@ -33,6 +59,14 @@ ZABBIX_PASS="Password"
NETBOX_HOST="https://netbox.local"
NETBOX_TOKEN="secrettoken"
```
Or, you can use a Zabbix API token to login instead of using a username and password.
In that case `ZABBIX_USER` and `ZABBIX_PASS` will be ignored.
```
ZABBIX_TOKEN=othersecrettoken
```
### Netbox custom fields
Use the following custom fields in Netbox (if you are using config context for the template information then the zabbix_template field is not required):
```
@@ -49,49 +83,84 @@ Use the following custom fields in Netbox (if you are using config context for t
* Default: null
* Object: dcim > device_type
```
You can make the hostID field hidden or read-only to prevent human intervention.
You can make the `zabbix_hostid` field hidden or read-only to prevent human intervention.
This is optional and there is a use case for leaving it read-write in the UI to manually change the ID. For example to re-run a sync.
## Virtual Machine (VM) Syncing
In order to use VM syncing, make sure that the `zabbix_id` custom field is also present on Virtual machine objects in Netbox.
Use the `config.py` file and set the `sync_vms` variable to `True`.
You can set the `vm_hostgroup_format` variable to a customizable value for VM hostgroups. The default is `cluster_type/cluster/role`.
To enable filtering for VM's, check the `nb_vm_filter` variable out. It works the same as with the device filter (see documentation under "Hostgroup layout"). Note that not all filtering capabilities and properties of devices are applicable to VM's and vice-versa. Check the Netbox API documentation to see which filtering options are available for each object type.
## Config file
### Hostgroup
Setting the create_hostgroups variable to False requires manual hostgroup creation for devices in a new category.
Setting the `create_hostgroups` variable to `False` requires manual hostgroup creation for devices in a new category. I would recommend setting this variable to `True` since leaving it on `False` results in a lot of manual work.
The format can be set with the hostgroup_format variable.
The format can be set with the `hostgroup_format` variable for devices and `vm_hostgroup_format` for devices.
Any nested parent hostgroups will also be created automatically. For instance the region `Berlin` with parent region `Germany` will create the hostgroup `Germany/Berlin`.
Make sure that the Zabbix user has proper permissions to create hosts.
The hostgroups are in a nested format. This means that proper permissions only need to be applied to the site name hostgroup and cascaded to any child hostgroups.
#### layout
#### Layout
The default hostgroup layout is "site/manufacturer/device_role".
**Variables**
You can change this behaviour with the hostgroup_format variable. The following values can be used:
**Both devices and virtual machines**
| name | description |
| ------------ | ------------ |
|dev_location|The device location name|
|dev_role|The device role name|
|manufacturer|Manufacturer name|
|region|The region name of the device|
|role|Role name of a device or VM|
|region|The region name|
|site|Site name|
|site_group|Site group name|
|tenant|Tenant name|
|tenant_group|Tenant group name|
|platform|Software platform of a device or VM|
|custom fields|See the section "Layout -> Custom Fields" to use custom fields as hostgroup variable|
**Only for devices**
| name | description |
| ------------ | ------------ |
|location|The device location name|
|manufacturer|Device manufacturer name|
**Only for VMs**
| name | description |
| ------------ | ------------ |
|cluster|VM cluster name|
|cluster_type|VM cluster type|
You can specify the value like so, sperated by a "/":
You can specify the value sperated by a "/" like so:
```
hostgroup_format = "tenant/site/dev_location/dev_role"
hostgroup_format = "tenant/site/dev_location/role"
```
**custom fields**
**Group traversal**
You can also use the value of custom fields under the device object.
The default behaviour for `region` is to only use the directly assigned region in the rendered hostgroup name.
However, by setting `traverse_region` to `True` in `config.py` the script will render a full region path of all parent regions for the hostgroup name.
`traverse_site_groups` controls the same behaviour for site_groups.
This allows more freedom and even allows a full static mapping instead of a dynamic rendered hostgroup name.
**Custom fields**
You can use the value of custom fields for hostgroup generation. This allows more freedom and even allows a full static mapping instead of a dynamic rendered hostgroup name.
For instance a custom field with the name `mycustomfieldname` and type string has the following values for 2 devices:
```
hostgroup_format = "site/mycustomfieldname"
Device A has the value Train for custom field mycustomfieldname.
Device B has the value Bus for custom field mycustomfieldname.
Both devices are located in the site Paris.
```
With the hostgroup format `site/mycustomfieldname` the following hostgroups will be generated:
```
Device A: Paris/Train
Device B: Paris/Bus
```
**Empty variables or hostgroups**
@@ -127,8 +196,27 @@ By setting a status on a Netbox device you determine how the host is added (or u
* Create the host in Zabbix with an enabled status (For now only enabled with the "Active" status)
You can modify this behaviour by changing the following list variables in the script:
- zabbix_device_removal
- zabbix_device_disable
- `zabbix_device_removal`
- `zabbix_device_disable`
### Zabbix Inventory
This script allows you to enable the inventory on managed Zabbix hosts and sync NetBox device properties to the specified inventory fields.
To map Netbox information to Netbox inventory fields, set `inventory_sync` to `True`.
You can set the inventory mode to "disabled", "manual" or "automatic" with the `inventory_mode` variable.
See [Zabbix Manual](https://www.zabbix.com/documentation/current/en/manual/config/hosts/inventory#building-inventory) for more information about the modes.
Use the `inventory_map` variable to map which NetBox properties are used in which Zabbix Inventory fields.
For nested properties, you can use the '/' seperator.
For example, the following map will assign the custom field 'mycustomfield' to the 'alias' Zabbix inventory field:
```
inventory_sync = True
inventory_mode = "manual"
inventory_map = { "custom_fields/mycustomfield/name": "alias"}
```
See `config.py.example` for an extensive example map.
Any Zabix Inventory fields that are not included in the map will not be touched by the script,
so you can safely add manual values or use items to automatically add values to other fields.
### Template source
You can either use a Netbox device type custom field or Netbox config context for the Zabbix template information.
@@ -174,8 +262,8 @@ If you want to automatically create hostgroups then the create permission on hos
To make the user experience easier you could add a custom link that redirects users to the Zabbix latest data.
```
* Name: zabbix_latestData
* Text: {% if obj.cf["zabbix_hostid"] %}Show host in Zabbix{% endif %}
* URL: http://myzabbixserver.local/zabbix.php?action=latest.view&hostids[]={{ obj.cf["zabbix_hostid"] }}
* Text: {% if object.cf["zabbix_hostid"] %}Show host in Zabbix{% endif %}
* URL: http://myzabbixserver.local/zabbix.php?action=latest.view&hostids[]={{ object.cf["zabbix_hostid"] }}
```
## Running the script
```
@@ -197,7 +285,29 @@ You can set the proxy for a device using the 'proxy' key in config context.
}
}
```
Because of the posible amount of destruction when setting up Netbox but forgetting the proxy command, the sync works a bit different. By default everything is synced except in a situation where the Zabbix host has a proxy configured but nothing is configured in Netbox. To force deletion and a full sync, set the full_proxy_sync variable in the config file.
It is now posible to specify proxy groups with the introduction of Proxy groups in Zabbix 7. Specifying a group in the config context on older Zabbix releases will have no impact and the script will ignore the statement.
```json
{
"zabbix": {
"proxy_group": "yourawesomeproxygroup.local"
}
}
```
The script will prefer groups when specifying both a proxy and group. This is done with the assumption that groups are more resiliant and HA ready, making it a more logical choice to use for proxy linkage. This also makes migrating from a proxy to proxy group easier since the group take priority over the invidivual proxy.
```json
{
"zabbix": {
"proxy": "yourawesomeproxy.local",
"proxy_group": "yourawesomeproxygroup.local"
}
}
```
In the example above the host will use the group on Zabbix 7. On Zabbix 6 and below the host will use the proxy. Zabbix 7 will use the proxy value when ommiting the proxy_group value.
Because of the possible amount of destruction when setting up Netbox but forgetting the proxy command, the sync works a bit different. By default everything is synced except in a situation where the Zabbix host has a proxy configured but nothing is configured in Netbox. To force deletion and a full sync, set the `full_proxy_sync` variable in the config file.
### Set interface parameters within Netbox
When adding a new device, you can set the interface type with custom context. By default, the following configuration is applied when no config context is provided:
@@ -263,4 +373,4 @@ To configure the interface parameters you'll need to use custom context. Custom
I would recommend using macros for sensitive data such as community strings since the data in Netbox is plain-text.
Note: Not all SNMP data is required for a working configuration. [The following parameters are allowed ](https://www.zabbix.com/documentation/current/manual/api/reference/hostinterface/object#details_tag "The following parameters are allowed ")but are not all required, depending on your environment.
> **_NOTE:_** Not all SNMP data is required for a working configuration. [The following parameters are allowed ](https://www.zabbix.com/documentation/current/manual/api/reference/hostinterface/object#details_tag "The following parameters are allowed ")but are not all required, depending on your environment.

View File

@@ -1,8 +1,9 @@
# Template logic.
## Template logic.
# Set to true to enable the template source information
# coming from config context instead of a custom field.
templates_config_context = False
# Set to true to give config context templates a
# Set to true to give config context templates a
# higher priority then custom field templates
templates_config_context_overrule = False
@@ -11,37 +12,87 @@ templates_config_context_overrule = False
template_cf = "zabbix_template"
device_cf = "zabbix_hostid"
# Enable clustering of devices with virtual chassis setup
## Enable clustering of devices with virtual chassis setup
clustering = False
# Enable hostgroup generation. Requires permissions in Zabbix
## Enable hostgroup generation. Requires permissions in Zabbix
create_hostgroups = True
# Create journal entries
## Create journal entries
create_journal = False
## Virtual machine sync
# Set sync_vms to True in order to use this new feature
# Use the hostgroup vm_hostgroup_format mapper for specific
# hostgroup atributes of VM's such as cluster_type and cluster
sync_vms = False
# Check the README documentation for values to use in the VM hostgroup format.
vm_hostgroup_format = "cluster_type/cluster/role"
## Proxy Sync
# Set to true to enable removal of proxy's under hosts. Use with caution and make sure that you specified
# all the required proxy's in the device config context before enabeling this option.
# With this option disabled proxy's will only be added and modified for Zabbix hosts.
full_proxy_sync = False
# Netbox to Zabbix device state convertion
## Netbox to Zabbix device state convertion
zabbix_device_removal = ["Decommissioning", "Inventory"]
zabbix_device_disable = ["Offline", "Planned", "Staged", "Failed"]
# Hostgroup mapping
# Available choices: dev_location, dev_role, manufacturer, region, site, site_group, tenant, tenant_group
## Hostgroup mapping
# See the README documentation for available options
# You can also use CF (custom field) names under the device. The CF content will be used for the hostgroup generation.
hostgroup_format = "site/manufacturer/dev_role"
#
# When using region in the group name, the default behaviour is to use name of the directly assigned region.
# By setting traverse_regions to True the full path of all parent regions will be used in the hostgroup, e.g.:
#
# 'Global/Europe/Netherlands/Amsterdam' instead of just 'Amsterdam'.
#
# traverse_site_groups controls the same behaviour for any assigned site_groups.
hostgroup_format = "site/manufacturer/role"
traverse_regions = False
traverse_site_groups = False
# Custom filter for device filtering. Variable must be present but can be left empty with no filtering.
# A couple of examples are as follows:
## Filtering
# Custom device filter, variable must be present but can be left empty with no filtering.
# A couple of examples:
# nb_device_filter = {} #No filter
# nb_device_filter = {"tag": "zabbix"} #Use a tag
# nb_device_filter = {"site": "HQ-AMS"} #Use a site name
# nb_device_filter = {"site": ["HQ-AMS", "HQ-FRA"]} #Device must be in either one of these sites
# nb_device_filter = {"site": "HQ-AMS", "tag": "zabbix", "role__n": ["PDU", "console-server"]} #Device must be in site HQ-AMS, have the tag zabbix and must not be part of the PDU or console-server role
# nb_device_filter = {} #No filter
# nb_device_filter = {"tag": "zabbix"} #Use a tag
# nb_device_filter = {"site": "HQ-AMS"} #Use a site name
# nb_device_filter = {"site": ["HQ-AMS", "HQ-FRA"]} #Device must be in either one of these sites
# nb_device_filter = {"site": "HQ-AMS", "tag": "zabbix", "role__n": ["PDU", "console-server"]} #Device must be in site HQ-AMS, have the tag zabbix and must not be part of the PDU or console-server role
# Default device filter, only get devices which have a name in Netbox:
nb_device_filter = {"name__n": "null"}
# Default filter for VMs
nb_vm_filter = {"name__n": "null"}
# Default device filter, only get devices which have a name in Netbox.
nb_device_filter = {"name__n": "null"}
## Inventory
# See https://www.zabbix.com/documentation/current/en/manual/config/hosts/inventory#building-inventory
# Choice between disabled, manual or automatic.
# Make sure to select at least manual or automatic in use with the inventory_sync function.
inventory_mode = "disabled"
# To allow syncing of NetBox device properties, set inventory_sync to True
inventory_sync = False
# inventory_map is used to map NetBox properties to Zabbix Inventory fields.
# For nested properties, you can use the '/' seperator.
# For example, the following map will assign the custom field 'mycustomfield' to the 'alias' Zabbix inventory field:
#
# inventory_map = { "custom_fields/mycustomfield/name": "alias"}
#
# The following map should provide some nice defaults:
inventory_map = { "asset_tag": "asset_tag",
"virtual_chassis/name": "chassis",
"status/label": "deployment_status",
"location/name": "location",
"latitude": "location_lat",
"longitude": "location_lon",
"comments": "notes",
"name": "name",
"rack/name": "site_rack",
"serial": "serialno_a",
"device_type/model": "type",
"device_type/manufacturer/name": "vendor",
"oob_ip/address": "oob_ip" }

774
modules/device.py Normal file
View File

@@ -0,0 +1,774 @@
#!/usr/bin/env python3
# pylint: disable=invalid-name, logging-not-lazy, too-many-locals, logging-fstring-interpolation, too-many-lines
"""
Device specific handeling for Netbox to Zabbix
"""
from os import sys
from re import search
from logging import getLogger
from zabbix_utils import APIRequestError
from modules.exceptions import (SyncInventoryError, TemplateError, SyncExternalError,
InterfaceConfigError, JournalError)
from modules.interface import ZabbixInterface
from modules.hostgroups import Hostgroup
try:
from config import (
template_cf, device_cf,
traverse_site_groups,
traverse_regions,
inventory_sync,
inventory_mode,
inventory_map
)
except ModuleNotFoundError:
print("Configuration file config.py not found in main directory."
"Please create the file or rename the config.py.example file to config.py.")
sys.exit(0)
class PhysicalDevice():
# pylint: disable=too-many-instance-attributes, too-many-arguments, too-many-positional-arguments
"""
Represents Network device.
INPUT: (Netbox device class, ZabbixAPI class, journal flag, NB journal class)
"""
def __init__(self, nb, zabbix, nb_journal_class, nb_version, journal=None, logger=None):
self.nb = nb
self.id = nb.id
self.name = nb.name
self.visible_name = None
self.status = nb.status.label
self.zabbix = zabbix
self.zabbix_id = None
self.group_id = None
self.nb_api_version = nb_version
self.zbx_template_names = []
self.zbx_templates = []
self.hostgroup = None
self.tenant = nb.tenant
self.config_context = nb.config_context
self.zbxproxy = None
self.zabbix_state = 0
self.journal = journal
self.nb_journals = nb_journal_class
self.inventory_mode = -1
self.inventory = {}
self.logger = logger if logger else getLogger(__name__)
self._setBasics()
def __repr__(self):
return self.name
def __str__(self):
return self.__repr__()
def _setBasics(self):
"""
Sets basic information like IP address.
"""
# Return error if device does not have primary IP.
if self.nb.primary_ip:
self.cidr = self.nb.primary_ip.address
self.ip = self.cidr.split("/")[0]
else:
e = f"Host {self.name}: no primary IP."
self.logger.info(e)
raise SyncInventoryError(e)
# Check if device has custom field for ZBX ID
if device_cf in self.nb.custom_fields:
self.zabbix_id = self.nb.custom_fields[device_cf]
else:
e = f"Host {self.name}: Custom field {device_cf} not present"
self.logger.warning(e)
raise SyncInventoryError(e)
# Validate hostname format.
odd_character_list = ["ä", "ö", "ü", "Ä", "Ö", "Ü", "ß"]
self.use_visible_name = False
if (any(letter in self.name for letter in odd_character_list) or
bool(search('[\u0400-\u04FF]', self.name))):
self.name = f"NETBOX_ID{self.id}"
self.visible_name = self.nb.name
self.use_visible_name = True
self.logger.info(f"Host {self.visible_name} contains special characters. "
f"Using {self.name} as name for the Netbox object "
f"and using {self.visible_name} as visible name in Zabbix.")
else:
pass
def set_hostgroup(self, hg_format, nb_site_groups, nb_regions):
"""Set the hostgroup for this device"""
# Create new Hostgroup instance
hg = Hostgroup("dev", self.nb, self.nb_api_version)
# Set Hostgroup nesting options
hg.set_nesting(traverse_site_groups, traverse_regions, nb_site_groups, nb_regions)
# Generate hostgroup based on hostgroup format
self.hostgroup = hg.generate(hg_format)
def set_template(self, prefer_config_context, overrule_custom):
""" Set Template """
self.zbx_template_names = None
# Gather templates ONLY from the device specific context
if prefer_config_context:
try:
self.zbx_template_names = self.get_templates_context()
except TemplateError as e:
self.logger.warning(e)
return True
# Gather templates from the custom field but overrule
# them should there be any device specific templates
if overrule_custom:
try:
self.zbx_template_names = self.get_templates_context()
except TemplateError:
pass
if not self.zbx_template_names:
self.zbx_template_names = self.get_templates_cf()
return True
# Gather templates ONLY from the custom field
self.zbx_template_names = self.get_templates_cf()
return True
def get_templates_cf(self):
""" Get template from custom field """
# Get Zabbix templates from the device type
device_type_cfs = self.nb.device_type.custom_fields
# Check if the ZBX Template CF is present
if template_cf in device_type_cfs:
# Set value to template
return [device_type_cfs[template_cf]]
# Custom field not found, return error
e = (f"Custom field {template_cf} not "
f"found for {self.nb.device_type.manufacturer.name}"
f" - {self.nb.device_type.display}.")
raise TemplateError(e)
def get_templates_context(self):
""" Get Zabbix templates from the device context """
if "zabbix" not in self.config_context:
e = (f"Host {self.name}: Key 'zabbix' not found in config "
"context for template lookup")
raise TemplateError(e)
if "templates" not in self.config_context["zabbix"]:
e = (f"Host {self.name}: Key 'templates' not found in config "
"context 'zabbix' for template lookup")
raise TemplateError(e)
return self.config_context["zabbix"]["templates"]
def set_inventory(self, nbdevice):
""" Set host inventory """
# Set inventory mode. Default is disabled (see class init function).
if inventory_mode == "disabled":
if inventory_sync:
self.logger.error(f"Host {self.name}: Unable to map Netbox inventory to Zabbix. "
"Inventory sync is enabled in config but inventory mode is disabled.")
return True
if inventory_mode == "manual":
self.inventory_mode = 0
elif inventory_mode == "automatic":
self.inventory_mode = 1
else:
self.logger.error(f"Host {self.name}: Specified value for inventory mode in"
f" config is not valid. Got value {inventory_mode}")
return False
self.inventory = {}
if inventory_sync and self.inventory_mode in [0,1]:
self.logger.debug(f"Host {self.name}: Starting inventory mapper")
# Let's build an inventory dict for each property in the inventory_map
for nb_inv_field, zbx_inv_field in inventory_map.items():
field_list = nb_inv_field.split("/") # convert str to list based on delimiter
# start at the base of the dict...
value = nbdevice
# ... and step through the dict till we find the needed value
for item in field_list:
value = value[item] if value else None
# Check if the result is usable and expected
# We want to apply any int or float 0 values,
# even if python thinks those are empty.
if ((value and isinstance(value, int | float | str )) or
(isinstance(value, int | float) and int(value) ==0)):
self.inventory[zbx_inv_field] = str(value)
elif not value:
# empty value should just be an empty string for API compatibility
self.logger.debug(f"Host {self.name}: Netbox inventory lookup for "
f"'{nb_inv_field}' returned an empty value")
self.inventory[zbx_inv_field] = ""
else:
# Value is not a string or numeral, probably not what the user expected.
self.logger.error(f"Host {self.name}: Inventory lookup for '{nb_inv_field}'"
" returned an unexpected type: it will be skipped.")
self.logger.debug(f"Host {self.name}: Inventory mapping complete. "
f"Mapped {len(list(filter(None, self.inventory.values())))} field(s)")
return True
def isCluster(self):
"""
Checks if device is part of cluster.
"""
return bool(self.nb.virtual_chassis)
def getClusterMaster(self):
"""
Returns chassis master ID.
"""
if not self.isCluster():
e = (f"Unable to proces {self.name} for cluster calculation: "
f"not part of a cluster.")
self.logger.warning(e)
raise SyncInventoryError(e)
if not self.nb.virtual_chassis.master:
e = (f"{self.name} is part of a Netbox virtual chassis which does "
"not have a master configured. Skipping for this reason.")
self.logger.error(e)
raise SyncInventoryError(e)
return self.nb.virtual_chassis.master.id
def promoteMasterDevice(self):
"""
If device is Primary in cluster,
promote device name to the cluster name.
Returns True if succesfull, returns False if device is secondary.
"""
masterid = self.getClusterMaster()
if masterid == self.id:
self.logger.debug(f"Host {self.name} is primary cluster member. "
f"Modifying hostname from {self.name} to " +
f"{self.nb.virtual_chassis.name}.")
self.name = self.nb.virtual_chassis.name
return True
self.logger.debug(f"Host {self.name} is non-primary cluster member.")
return False
def zbxTemplatePrepper(self, templates):
"""
Returns Zabbix template IDs
INPUT: list of templates from Zabbix
OUTPUT: True
"""
# Check if there are templates defined
if not self.zbx_template_names:
e = f"Host {self.name}: No templates found"
self.logger.info(e)
raise SyncInventoryError()
# Set variable to empty list
self.zbx_templates = []
# Go through all templates definded in Netbox
for nb_template in self.zbx_template_names:
template_match = False
# Go through all templates found in Zabbix
for zbx_template in templates:
# If the template names match
if zbx_template['name'] == nb_template:
# Set match variable to true, add template details
# to class variable and return debug log
template_match = True
self.zbx_templates.append({"templateid": zbx_template['templateid'],
"name": zbx_template['name']})
e = f"Host {self.name}: found template {zbx_template['name']}"
self.logger.debug(e)
# Return error should the template not be found in Zabbix
if not template_match:
e = (f"Unable to find template {nb_template} "
f"for host {self.name} in Zabbix. Skipping host...")
self.logger.warning(e)
raise SyncInventoryError(e)
def setZabbixGroupID(self, groups):
"""
Sets Zabbix group ID as instance variable
INPUT: list of hostgroups
OUTPUT: True / False
"""
# Go through all groups
for group in groups:
if group['name'] == self.hostgroup:
self.group_id = group['groupid']
e = f"Host {self.name}: matched group {group['name']}"
self.logger.debug(e)
return True
return False
def cleanup(self):
"""
Removes device from external resources.
Resets custom fields in Netbox.
"""
if self.zabbix_id:
try:
# Check if the Zabbix host exists in Zabbix
zbx_host = bool(self.zabbix.host.get(filter={'hostid': self.zabbix_id},
output=[]))
e = (f"Host {self.name}: was already deleted from Zabbix."
" Removed link in Netbox.")
if zbx_host:
# Delete host should it exists
self.zabbix.host.delete(self.zabbix_id)
e = f"Host {self.name}: Deleted host from Zabbix."
self._zeroize_cf()
self.logger.info(e)
self.create_journal_entry("warning", "Deleted host from Zabbix")
except APIRequestError as e:
message = f"Zabbix returned the following error: {str(e)}."
self.logger.error(message)
raise SyncExternalError(message) from e
def _zeroize_cf(self):
"""Sets the hostID custom field in Netbox to zero,
effectively destroying the link"""
self.nb.custom_fields[device_cf] = None
self.nb.save()
def _zabbixHostnameExists(self):
"""
Checks if hostname exists in Zabbix.
"""
# Validate the hostname or visible name field
if not self.use_visible_name:
zbx_filter = {'host': self.name}
else:
zbx_filter = {'name': self.visible_name}
host = self.zabbix.host.get(filter=zbx_filter, output=[])
return bool(host)
def setInterfaceDetails(self):
"""
Checks interface parameters from Netbox and
creates a model for the interface to be used in Zabbix.
"""
try:
# Initiate interface class
interface = ZabbixInterface(self.nb.config_context, self.ip)
# Check if Netbox has device context.
# If not fall back to old config.
if interface.get_context():
# If device is SNMP type, add aditional information.
if interface.interface["type"] == 2:
interface.set_snmp()
else:
interface.set_default_snmp()
return [interface.interface]
except InterfaceConfigError as e:
message = f"{self.name}: {e}"
self.logger.warning(message)
raise SyncInventoryError(message) from e
def setProxy(self, proxy_list):
"""
Sets proxy or proxy group if this
value has been defined in config context
input: List of all proxies and proxy groups in standardized format
"""
# check if the key Zabbix is defined in the config context
if not "zabbix" in self.nb.config_context:
return False
if ("proxy" in self.nb.config_context["zabbix"] and
not self.nb.config_context["zabbix"]["proxy"]):
return False
# Proxy group takes priority over a proxy due
# to it being HA and therefore being more reliable
# Includes proxy group fix since Zabbix <= 6 should ignore this
proxy_types = ["proxy"]
if str(self.zabbix.version).startswith('7'):
# Only insert groups in front of list for Zabbix7
proxy_types.insert(0, "proxy_group")
for proxy_type in proxy_types:
# Check if the key exists in Netbox CC
if proxy_type in self.nb.config_context["zabbix"]:
proxy_name = self.nb.config_context["zabbix"][proxy_type]
# go through all proxies
for proxy in proxy_list:
# If the proxy does not match the type, ignore and continue
if not proxy["type"] == proxy_type:
continue
# If the proxy name matches
if proxy["name"] == proxy_name:
self.logger.debug(f"Host {self.name}: using {proxy['type']}"
f" {proxy_name}")
self.zbxproxy = proxy
return True
self.logger.warning(f"Host {self.name}: unable to find proxy {proxy_name}")
return False
def createInZabbix(self, groups, templates, proxies,
description="Host added by Netbox sync script."):
"""
Creates Zabbix host object with parameters from Netbox object.
"""
# Check if hostname is already present in Zabbix
if not self._zabbixHostnameExists():
# Set group and template ID's for host
if not self.setZabbixGroupID(groups):
e = (f"Unable to find group '{self.hostgroup}' "
f"for host {self.name} in Zabbix.")
self.logger.warning(e)
raise SyncInventoryError(e)
self.zbxTemplatePrepper(templates)
templateids = []
for template in self.zbx_templates:
templateids.append({'templateid': template['templateid']})
# Set interface, group and template configuration
interfaces = self.setInterfaceDetails()
groups = [{"groupid": self.group_id}]
# Set Zabbix proxy if defined
self.setProxy(proxies)
# Set basic data for host creation
create_data = {"host": self.name,
"name": self.visible_name,
"status": self.zabbix_state,
"interfaces": interfaces,
"groups": groups,
"templates": templateids,
"description": description,
"inventory_mode": self.inventory_mode,
"inventory": self.inventory
}
# If a Zabbix proxy or Zabbix Proxy group has been defined
if self.zbxproxy:
# If a lower version than 7 is used, we can assume that
# the proxy is a normal proxy and not a proxy group
if not str(self.zabbix.version).startswith('7'):
create_data["proxy_hostid"] = self.zbxproxy["id"]
else:
# Configure either a proxy or proxy group
create_data[self.zbxproxy["idtype"]] = self.zbxproxy["id"]
create_data["monitored_by"] = self.zbxproxy["monitored_by"]
# Add host to Zabbix
try:
host = self.zabbix.host.create(**create_data)
self.zabbix_id = host["hostids"][0]
except APIRequestError as e:
e = f"Host {self.name}: Couldn't create. Zabbix returned {str(e)}."
self.logger.error(e)
raise SyncExternalError(e) from None
# Set Netbox custom field to hostID value.
self.nb.custom_fields[device_cf] = int(self.zabbix_id)
self.nb.save()
msg = f"Host {self.name}: Created host in Zabbix."
self.logger.info(msg)
self.create_journal_entry("success", msg)
else:
e = f"Host {self.name}: Unable to add to Zabbix. Host already present."
self.logger.warning(e)
def createZabbixHostgroup(self, hostgroups):
"""
Creates Zabbix host group based on hostgroup format.
Creates multiple when using a nested format.
"""
final_data = []
# Check if the hostgroup is in a nested format and check each parent
for pos in range(len(self.hostgroup.split('/'))):
zabbix_hg = self.hostgroup.rsplit('/', pos)[0]
if self.lookupZabbixHostgroup(hostgroups, zabbix_hg):
# Hostgroup already exists
continue
# Create new group
try:
# API call to Zabbix
groupid = self.zabbix.hostgroup.create(name=zabbix_hg)
e = f"Hostgroup '{zabbix_hg}': created in Zabbix."
self.logger.info(e)
# Add group to final data
final_data.append({'groupid': groupid["groupids"][0], 'name': zabbix_hg})
except APIRequestError as e:
msg = f"Hostgroup '{zabbix_hg}': unable to create. Zabbix returned {str(e)}."
self.logger.error(msg)
raise SyncExternalError(msg) from e
return final_data
def lookupZabbixHostgroup(self, group_list, lookup_group):
"""
Function to check if a hostgroup
exists in a list of Zabbix hostgroups
INPUT: Group list and group lookup
OUTPUT: Boolean
"""
for group in group_list:
if group["name"] == lookup_group:
return True
return False
def updateZabbixHost(self, **kwargs):
"""
Updates Zabbix host with given parameters.
INPUT: Key word arguments for Zabbix host object.
"""
try:
self.zabbix.host.update(hostid=self.zabbix_id, **kwargs)
except APIRequestError as e:
e = (f"Host {self.name}: Unable to update. "
f"Zabbix returned the following error: {str(e)}.")
self.logger.error(e)
raise SyncExternalError(e) from None
self.logger.info(f"Updated host {self.name} with data {kwargs}.")
self.create_journal_entry("info", "Updated host in Zabbix with latest NB data.")
def ConsistencyCheck(self, groups, templates, proxies, proxy_power, create_hostgroups):
# pylint: disable=too-many-branches, too-many-statements
"""
Checks if Zabbix object is still valid with Netbox parameters.
"""
# If group is found or if the hostgroup is nested
if not self.setZabbixGroupID(groups) or len(self.hostgroup.split('/')) > 1:
if create_hostgroups:
# Script is allowed to create a new hostgroup
new_groups = self.createZabbixHostgroup(groups)
for group in new_groups:
# Add all new groups to the list of groups
groups.append(group)
# check if the initial group was not already found (and this is a nested folder check)
if not self.group_id:
# Function returns true / false but also sets GroupID
if not self.setZabbixGroupID(groups) and not create_hostgroups:
e = (f"Host {self.name}: different hostgroup is required but "
"unable to create hostgroup without generation permission.")
self.logger.warning(e)
raise SyncInventoryError(e)
# Prepare templates and proxy config
self.zbxTemplatePrepper(templates)
self.setProxy(proxies)
# Get host object from Zabbix
host = self.zabbix.host.get(filter={'hostid': self.zabbix_id},
selectInterfaces=['type', 'ip',
'port', 'details',
'interfaceid'],
selectGroups=["groupid"],
selectParentTemplates=["templateid"],
selectInventory=list(inventory_map.values()))
if len(host) > 1:
e = (f"Got {len(host)} results for Zabbix hosts "
f"with ID {self.zabbix_id} - hostname {self.name}.")
self.logger.error(e)
raise SyncInventoryError(e)
if len(host) == 0:
e = (f"Host {self.name}: No Zabbix host found. "
f"This is likely the result of a deleted Zabbix host "
f"without zeroing the ID field in Netbox.")
self.logger.error(e)
raise SyncInventoryError(e)
host = host[0]
if host["host"] == self.name:
self.logger.debug(f"Host {self.name}: hostname in-sync.")
else:
self.logger.warning(f"Host {self.name}: hostname OUT of sync. "
f"Received value: {host['host']}")
self.updateZabbixHost(host=self.name)
# Execute check depending on wether the name is special or not
if self.use_visible_name:
if host["name"] == self.visible_name:
self.logger.debug(f"Host {self.name}: visible name in-sync.")
else:
self.logger.warning(f"Host {self.name}: visible name OUT of sync."
f" Received value: {host['name']}")
self.updateZabbixHost(name=self.visible_name)
# Check if the templates are in-sync
if not self.zbx_template_comparer(host["parentTemplates"]):
self.logger.warning(f"Host {self.name}: template(s) OUT of sync.")
# Prepare Templates for API parsing
templateids = []
for template in self.zbx_templates:
templateids.append({'templateid': template['templateid']})
# Update Zabbix with NB templates and clear any old / lost templates
self.updateZabbixHost(templates_clear=host["parentTemplates"],
templates=templateids)
else:
self.logger.debug(f"Host {self.name}: template(s) in-sync.")
for group in host["groups"]:
if group["groupid"] == self.group_id:
self.logger.debug(f"Host {self.name}: hostgroup in-sync.")
break
else:
self.logger.warning(f"Host {self.name}: hostgroup OUT of sync.")
self.updateZabbixHost(groups={'groupid': self.group_id})
if int(host["status"]) == self.zabbix_state:
self.logger.debug(f"Host {self.name}: status in-sync.")
else:
self.logger.warning(f"Host {self.name}: status OUT of sync.")
self.updateZabbixHost(status=str(self.zabbix_state))
# Check if a proxy has been defined
if self.zbxproxy:
# Check if proxy or proxy group is defined
if (self.zbxproxy["idtype"] in host and
host[self.zbxproxy["idtype"]] == self.zbxproxy["id"]):
self.logger.debug(f"Host {self.name}: proxy in-sync.")
# Backwards compatibility for Zabbix <= 6
elif "proxy_hostid" in host and host["proxy_hostid"] == self.zbxproxy["id"]:
self.logger.debug(f"Host {self.name}: proxy in-sync.")
# Proxy does not match, update Zabbix
else:
self.logger.warning(f"Host {self.name}: proxy OUT of sync.")
# Zabbix <= 6 patch
if not str(self.zabbix.version).startswith('7'):
self.updateZabbixHost(proxy_hostid=self.zbxproxy['id'])
# Zabbix 7+
else:
# Prepare data structure for updating either proxy or group
update_data = {self.zbxproxy["idtype"]: self.zbxproxy["id"],
"monitored_by": self.zbxproxy['monitored_by']}
self.updateZabbixHost(**update_data)
else:
# No proxy is defined in Netbox
proxy_set = False
# Check if a proxy is defined. Uses the proxy_hostid key for backwards compatibility
for key in ("proxy_hostid", "proxyid", "proxy_groupid"):
if key in host:
if bool(int(host[key])):
proxy_set = True
if proxy_power and proxy_set:
# Zabbix <= 6 fix
self.logger.warning(f"Host {self.name}: no proxy is configured in Netbox "
"but is configured in Zabbix. Removing proxy config in Zabbix")
if "proxy_hostid" in host and bool(host["proxy_hostid"]):
self.updateZabbixHost(proxy_hostid=0)
# Zabbix 7 proxy
elif "proxyid" in host and bool(host["proxyid"]):
self.updateZabbixHost(proxyid=0, monitored_by=0)
# Zabbix 7 proxy group
elif "proxy_groupid" in host and bool(host["proxy_groupid"]):
self.updateZabbixHost(proxy_groupid=0, monitored_by=0)
# Checks if a proxy has been defined in Zabbix and if proxy_power config has been set
if proxy_set and not proxy_power:
# Display error message
self.logger.error(f"Host {self.name} is configured "
f"with proxy in Zabbix but not in Netbox. The"
" -p flag was ommited: no "
"changes have been made.")
if not proxy_set:
self.logger.debug(f"Host {self.name}: proxy in-sync.")
# Check host inventory mode
if str(host['inventory_mode']) == str(self.inventory_mode):
self.logger.debug(f"Host {self.name}: inventory_mode in-sync.")
else:
self.logger.warning(f"Host {self.name}: inventory_mode OUT of sync.")
self.updateZabbixHost(inventory_mode=str(self.inventory_mode))
if inventory_sync and self.inventory_mode in [0,1]:
# Check host inventory mapping
if host['inventory'] == self.inventory:
self.logger.debug(f"Host {self.name}: inventory in-sync.")
else:
self.logger.warning(f"Host {self.name}: inventory OUT of sync.")
self.updateZabbixHost(inventory=self.inventory)
# If only 1 interface has been found
# pylint: disable=too-many-nested-blocks
if len(host['interfaces']) == 1:
updates = {}
# Go through each key / item and check if it matches Zabbix
for key, item in self.setInterfaceDetails()[0].items():
# Check if Netbox value is found in Zabbix
if key in host["interfaces"][0]:
# If SNMP is used, go through nested dict
# to compare SNMP parameters
if isinstance(item,dict) and key == "details":
for k, i in item.items():
if k in host["interfaces"][0][key]:
# Set update if values don't match
if host["interfaces"][0][key][k] != str(i):
# If dict has not been created, add it
if key not in updates:
updates[key] = {}
updates[key][k] = str(i)
# If SNMP version has been changed
# break loop and force full SNMP update
if k == "version":
break
# Force full SNMP config update
# when version has changed.
if key in updates:
if "version" in updates[key]:
for k, i in item.items():
updates[key][k] = str(i)
continue
# Set update if values don't match
if host["interfaces"][0][key] != str(item):
updates[key] = item
if updates:
# If interface updates have been found: push to Zabbix
self.logger.warning(f"Host {self.name}: Interface OUT of sync.")
if "type" in updates:
# Changing interface type not supported. Raise exception.
e = (f"Host {self.name}: changing interface type to "
f"{str(updates['type'])} is not supported.")
self.logger.error(e)
raise InterfaceConfigError(e)
# Set interfaceID for Zabbix config
updates["interfaceid"] = host["interfaces"][0]['interfaceid']
try:
# API call to Zabbix
self.zabbix.hostinterface.update(updates)
e = f"Host {self.name}: solved interface conflict."
self.logger.info(e)
self.create_journal_entry("info", e)
except APIRequestError as e:
msg = f"Zabbix returned the following error: {str(e)}."
self.logger.error(msg)
raise SyncExternalError(msg) from e
else:
# If no updates are found, Zabbix interface is in-sync
e = f"Host {self.name}: interface in-sync."
self.logger.debug(e)
else:
e = (f"Host {self.name} has unsupported interface configuration."
f" Host has total of {len(host['interfaces'])} interfaces. "
"Manual interfention required.")
self.logger.error(e)
raise SyncInventoryError(e)
def create_journal_entry(self, severity, message):
"""
Send a new Journal entry to Netbox. Usefull for viewing actions
in Netbox without having to look in Zabbix or the script log output
"""
if self.journal:
# Check if the severity is valid
if severity not in ["info", "success", "warning", "danger"]:
self.logger.warning(f"Value {severity} not valid for NB journal entries.")
return False
journal = {"assigned_object_type": "dcim.device",
"assigned_object_id": self.id,
"kind": severity,
"comments": message
}
try:
self.nb_journals.create(journal)
self.logger.debug(f"Host {self.name}: Created journal entry in Netbox")
return True
except JournalError(e) as e:
self.logger.warning("Unable to create journal entry for "
f"{self.name}: NB returned {e}")
return False
return False
def zbx_template_comparer(self, tmpls_from_zabbix):
"""
Compares the Netbox and Zabbix templates with each other.
Should there be a mismatch then the function will return false
INPUT: list of NB and ZBX templates
OUTPUT: Boolean True/False
"""
succesfull_templates = []
# Go through each Netbox template
for nb_tmpl in self.zbx_templates:
# Go through each Zabbix template
for pos, zbx_tmpl in enumerate(tmpls_from_zabbix):
# Check if template IDs match
if nb_tmpl["templateid"] == zbx_tmpl["templateid"]:
# Templates match. Remove this template from the Zabbix templates
# and add this NB template to the list of successfull templates
tmpls_from_zabbix.pop(pos)
succesfull_templates.append(nb_tmpl)
self.logger.debug(f"Host {self.name}: template "
f"{nb_tmpl['name']} is present in Zabbix.")
break
if len(succesfull_templates) == len(self.zbx_templates) and len(tmpls_from_zabbix) == 0:
# All of the Netbox templates have been confirmed as successfull
# and the ZBX template list is empty. This means that
# all of the templates match.
return True
return False

33
modules/exceptions.py Normal file
View File

@@ -0,0 +1,33 @@
#!/usr/bin/env python3
"""
All custom exceptions used for Exception generation
"""
class SyncError(Exception):
""" Class SyncError """
class JournalError(Exception):
""" Class SyncError """
class SyncExternalError(SyncError):
""" Class SyncExternalError """
class SyncInventoryError(SyncError):
""" Class SyncInventoryError """
class SyncDuplicateError(SyncError):
""" Class SyncDuplicateError """
class EnvironmentVarError(SyncError):
""" Class EnvironmentVarError """
class InterfaceConfigError(SyncError):
""" Class InterfaceConfigError """
class ProxyConfigError(SyncError):
""" Class ProxyConfigError """
class HostgroupError(SyncError):
""" Class HostgroupError """
class TemplateError(SyncError):
""" Class TemplateError """

160
modules/hostgroups.py Normal file
View File

@@ -0,0 +1,160 @@
"""Module for all hostgroup related code"""
from logging import getLogger
from modules.exceptions import HostgroupError
from modules.tools import build_path
class Hostgroup():
"""Hostgroup class for devices and VM's
Takes type (vm or dev) and NB object"""
def __init__(self, obj_type, nb_obj, version, logger=None):
self.logger = logger if logger else getLogger(__name__)
if obj_type not in ("vm", "dev"):
msg = f"Unable to create hostgroup with type {type}"
self.logger.error()
raise HostgroupError(msg)
self.type = str(obj_type)
self.nb = nb_obj
self.name = self.nb.name
self.nb_version = version
# Used for nested data objects
self.nested_objects = {}
self._set_format_options()
def __str__(self):
return f"Hostgroup for {self.type} {self.name}"
def __repr__(self):
return self.__str__()
def _set_format_options(self):
"""
Set all available variables
for hostgroup generation
"""
format_options = {}
# Set variables for both type of devices
if self.type in ("vm", "dev"):
# Role fix for Netbox <=3
role = None
if self.nb_version.startswith(("2", "3")) and self.type == "dev":
role = self.nb.device_role.name if self.nb.device_role else None
else:
role = self.nb.role.name if self.nb.role else None
# Add default formatting options
# Check if a site is configured. A site is optional for VMs
format_options["region"] = None
format_options["site_group"] = None
if self.nb.site:
if self.nb.site.region:
format_options["region"] = self.generate_parents("region",
str(self.nb.site.region))
if self.nb.site.group:
format_options["site_group"] = self.generate_parents("site_group",
str(self.nb.site.group))
format_options["role"] = role
format_options["site"] = self.nb.site.name if self.nb.site else None
format_options["tenant"] = str(self.nb.tenant) if self.nb.tenant else None
format_options["tenant_group"] = str(self.nb.tenant.group) if self.nb.tenant else None
format_options["platform"] = self.nb.platform.name if self.nb.platform else None
# Variables only applicable for devices
if self.type == "dev":
format_options["manufacturer"] = self.nb.device_type.manufacturer.name
format_options["location"] = str(self.nb.location) if self.nb.location else None
# Variables only applicable for VM's
if self.type == "vm":
# Check if a cluster is configured. Could also be configured in a site.
if self.nb.cluster:
format_options["cluster"] = self.nb.cluster.name
format_options["cluster_type"] = self.nb.cluster.type.name
self.format_options = format_options
def set_nesting(self, nested_sitegroup_flag, nested_region_flag,
nb_groups, nb_regions):
"""Set nesting options for this Hostgroup"""
self.nested_objects = {"site_group": {"flag": nested_sitegroup_flag, "data": nb_groups},
"region": {"flag": nested_region_flag, "data": nb_regions}}
def generate(self, hg_format=None):
"""Generate hostgroup based on a provided format"""
# Set format to default in case its not specified
if not hg_format:
hg_format = "site/manufacturer/role" if self.type == "dev" else "cluster/role"
# Split all given names
hg_output = []
hg_items = hg_format.split("/")
for hg_item in hg_items:
# Check if requested data is available as option for this host
if hg_item not in self.format_options:
# Check if a custom field exists with this name
cf_data = self.custom_field_lookup(hg_item)
# CF does not exist
if not cf_data["result"]:
msg = (f"Unable to generate hostgroup for host {self.name}. "
f"Item type {hg_item} not supported.")
self.logger.error(msg)
raise HostgroupError(msg)
# CF data is populated
if cf_data["cf"]:
hg_output.append(cf_data["cf"])
continue
# Check if there is a value associated to the variable.
# For instance, if a device has no location, do not use it with hostgroup calculation
hostgroup_value = self.format_options[hg_item]
if hostgroup_value:
hg_output.append(hostgroup_value)
# Check if the hostgroup is populated with at least one item.
if bool(hg_output):
return "/".join(hg_output)
msg = (f"Unable to generate hostgroup for host {self.name}."
" Not enough valid items. This is most likely"
" due to the use of custom fields that are empty"
" or an invalid hostgroup format.")
self.logger.error(msg)
raise HostgroupError(msg)
def list_formatoptions(self):
"""
Function to easily troubleshoot which values
are generated for a specific device or VM.
"""
print(f"The following options are available for host {self.name}")
for option_type, value in self.format_options.items():
if value is not None:
print(f"{option_type} - {value}")
print("The following options are not available")
for option_type, value in self.format_options.items():
if value is None:
print(f"{option_type}")
def custom_field_lookup(self, hg_category):
"""
Checks if a valid custom field is present in Netbox.
INPUT: Custom field name
OUTPUT: dictionary with 'result' and 'cf' keys.
"""
# Check if the custom field exists
if hg_category not in self.nb.custom_fields:
return {"result": False, "cf": None}
# Checks if the custom field has been populated
if not bool(self.nb.custom_fields[hg_category]):
return {"result": True, "cf": None}
# Custom field exists and is populated
return {"result": True, "cf": self.nb.custom_fields[hg_category]}
def generate_parents(self, nest_type, child_object):
"""
Generates parent objects to implement nested regions / nested site groups
INPUT: nest_type to set which type of nesting is going to be processed
child_object: the name of the child object (for instance the last NB region)
OUTPUT: STRING - Either the single child name or child and parents.
"""
# Check if this type of nesting is supported.
if not nest_type in self.nested_objects:
return child_object
# If the nested flag is True, perform parent calculation
if self.nested_objects[nest_type]["flag"]:
final_nested_object = build_path(child_object, self.nested_objects[nest_type]["data"])
return "/".join(final_nested_object)
# Nesting is not allowed for this object. Return child_object
return child_object

105
modules/interface.py Normal file
View File

@@ -0,0 +1,105 @@
#!/usr/bin/env python3
"""
All of the Zabbix interface related configuration
"""
from modules.exceptions import InterfaceConfigError
class ZabbixInterface():
"""Class that represents a Zabbix interface."""
def __init__(self, context, ip):
self.context = context
self.ip = ip
self.skelet = {"main": "1", "useip": "1", "dns": "", "ip": self.ip}
self.interface = self.skelet
def _set_default_port(self):
"""Sets default TCP / UDP port for different interface types"""
interface_mapping = {
1: 10050,
2: 161,
3: 623,
4: 12345
}
# Check if interface type is listed in mapper.
if self.interface['type'] not in interface_mapping:
return False
# Set default port to interface
self.interface["port"] = str(interface_mapping[self.interface['type']])
return True
def get_context(self):
""" check if Netbox custom context has been defined. """
if "zabbix" in self.context:
zabbix = self.context["zabbix"]
if "interface_type" in zabbix:
self.interface["type"] = zabbix["interface_type"]
if not "interface_port" in zabbix:
self._set_default_port()
return True
self.interface["port"] = zabbix["interface_port"]
return True
return False
return False
def set_snmp(self):
""" Check if interface is type SNMP """
# pylint: disable=too-many-branches
if self.interface["type"] == 2:
# Checks if SNMP settings are defined in Netbox
if "snmp" in self.context["zabbix"]:
snmp = self.context["zabbix"]["snmp"]
self.interface["details"] = {}
# Checks if bulk config has been defined
if "bulk" in snmp:
self.interface["details"]["bulk"] = str(snmp.pop("bulk"))
else:
# Fallback to bulk enabled if not specified
self.interface["details"]["bulk"] = "1"
# SNMP Version config is required in Netbox config context
if snmp.get("version"):
self.interface["details"]["version"] = str(snmp.pop("version"))
else:
e = "SNMP version option is not defined."
raise InterfaceConfigError(e)
# If version 1 or 2 is used, get community string
if self.interface["details"]["version"] in ['1','2']:
if "community" in snmp:
# Set SNMP community to confix context value
community = snmp["community"]
else:
# Set SNMP community to default
community = "{$SNMP_COMMUNITY}"
self.interface["details"]["community"] = str(community)
# If version 3 has been used, get all
# SNMPv3 Netbox related configs
elif self.interface["details"]["version"] == '3':
items = ["securityname", "securitylevel", "authpassphrase",
"privpassphrase", "authprotocol", "privprotocol",
"contextname"]
for key, item in snmp.items():
if key in items:
self.interface["details"][key] = str(item)
else:
e = "Unsupported SNMP version."
raise InterfaceConfigError(e)
else:
e = "Interface type SNMP but no parameters provided."
raise InterfaceConfigError(e)
else:
e = "Interface type is not SNMP, unable to set SNMP details"
raise InterfaceConfigError(e)
def set_default_snmp(self):
""" Set default config to SNMPv2, port 161 and community macro. """
self.interface = self.skelet
self.interface["type"] = "2"
self.interface["port"] = "161"
self.interface["details"] = {"version": "2",
"community": "{$SNMP_COMMUNITY}",
"bulk": "1"}
def set_default_agent(self):
"""Sets interface to Zabbix agent defaults"""
self.interface["type"] = "1"
self.interface["port"] = "10050"

44
modules/tools.py Normal file
View File

@@ -0,0 +1,44 @@
"""A collection of tools used by several classes"""
def convert_recordset(recordset):
""" Converts netbox RedcordSet to list of dicts. """
recordlist = []
for record in recordset:
recordlist.append(record.__dict__)
return recordlist
def build_path(endpoint, list_of_dicts):
"""
Builds a path list of related parent/child items.
This can be used to generate a joinable list to
be used in hostgroups.
"""
item_path = []
itemlist = [i for i in list_of_dicts if i['name'] == endpoint]
item = itemlist[0] if len(itemlist) == 1 else None
item_path.append(item['name'])
while item['_depth'] > 0:
itemlist = [i for i in list_of_dicts if i['name'] == str(item['parent'])]
item = itemlist[0] if len(itemlist) == 1 else None
item_path.append(item['name'])
item_path.reverse()
return item_path
def proxy_prepper(proxy_list, proxy_group_list):
"""
Function that takes 2 lists and converts them using a
standardized format for further processing.
"""
output = []
for proxy in proxy_list:
proxy["type"] = "proxy"
proxy["id"] = proxy["proxyid"]
proxy["idtype"] = "proxyid"
proxy["monitored_by"] = 1
output.append(proxy)
for group in proxy_group_list:
group["type"] = "proxy_group"
group["id"] = group["proxy_groupid"]
group["idtype"] = "proxy_groupid"
group["monitored_by"] = 2
output.append(group)
return output

View File

@@ -0,0 +1,65 @@
#!/usr/bin/env python3
# pylint: disable=duplicate-code
"""Module that hosts all functions for virtual machine processing"""
from os import sys
from modules.device import PhysicalDevice
from modules.hostgroups import Hostgroup
from modules.interface import ZabbixInterface
from modules.exceptions import TemplateError, InterfaceConfigError, SyncInventoryError
try:
from config import (
traverse_site_groups,
traverse_regions
)
except ModuleNotFoundError:
print("Configuration file config.py not found in main directory."
"Please create the file or rename the config.py.example file to config.py.")
sys.exit(0)
class VirtualMachine(PhysicalDevice):
"""Model for virtual machines"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.hostgroup = None
self.zbx_template_names = None
def set_hostgroup(self, hg_format, nb_site_groups, nb_regions):
"""Set the hostgroup for this device"""
# Create new Hostgroup instance
hg = Hostgroup("vm", self.nb, self.nb_api_version, logger=self.logger)
hg.set_nesting(traverse_site_groups, traverse_regions, nb_site_groups, nb_regions)
# Generate hostgroup based on hostgroup format
self.hostgroup = hg.generate(hg_format)
def set_vm_template(self):
""" Set Template for VMs. Overwrites default class
to skip a lookup of custom fields."""
# Gather templates ONLY from the device specific context
try:
self.zbx_template_names = self.get_templates_context()
except TemplateError as e:
self.logger.warning(e)
return True
def setInterfaceDetails(self): # pylint: disable=invalid-name
"""
Overwrites device function to select an agent interface type by default
Agent type interfaces are more likely to be used with VMs then SNMP
"""
try:
# Initiate interface class
interface = ZabbixInterface(self.nb.config_context, self.ip)
# Check if Netbox has device context.
# If not fall back to old config.
if interface.get_context():
# If device is SNMP type, add aditional information.
if interface.interface["type"] == 2:
interface.set_snmp()
else:
interface.set_default_agent()
return [interface.interface]
except InterfaceConfigError as e:
message = f"{self.name}: {e}"
self.logger.warning(message)
raise SyncInventoryError(message) from e

View File

@@ -1,28 +1,36 @@
#!/usr/bin/python3
"""Netbox to Zabbix sync script."""
#!/usr/bin/env python3
# pylint: disable=invalid-name, logging-not-lazy, too-many-locals, logging-fstring-interpolation
from os import environ, path, sys
"""Netbox to Zabbix sync script."""
import logging
import argparse
from os import environ, path, sys
from pynetbox import api
from pyzabbix import ZabbixAPI, ZabbixAPIException
from pynetbox.core.query import RequestError as NBRequestError
from requests.exceptions import ConnectionError as RequestsConnectionError
from zabbix_utils import ZabbixAPI, APIRequestError, ProcessingError
from modules.device import PhysicalDevice
from modules.virtual_machine import VirtualMachine
from modules.tools import convert_recordset, proxy_prepper
from modules.exceptions import EnvironmentVarError, HostgroupError, SyncError
try:
from config import (
templates_config_context,
templates_config_context_overrule,
clustering, create_hostgroups,
create_journal, full_proxy_sync,
template_cf, device_cf,
zabbix_device_removal,
zabbix_device_disable,
hostgroup_format,
nb_device_filter
vm_hostgroup_format,
nb_device_filter,
sync_vms,
nb_vm_filter
)
except ModuleNotFoundError:
print(f"Configuration file config.py not found in main directory."
print("Configuration file config.py not found in main directory."
"Please create the file or rename the config.py.example file to config.py.")
sys.exit(0)
sys.exit(1)
# Set logging
log_format = logging.Formatter('%(asctime)s - %(name)s - '
@@ -41,795 +49,221 @@ logger.addHandler(lgout)
logger.addHandler(lgfile)
logger.setLevel(logging.WARNING)
def main(arguments):
"""Run the sync process."""
# pylint: disable=too-many-branches, too-many-statements
# set environment variables
if(arguments.verbose):
if arguments.verbose:
logger.setLevel(logging.DEBUG)
env_vars = ["ZABBIX_HOST", "ZABBIX_USER", "ZABBIX_PASS",
"NETBOX_HOST", "NETBOX_TOKEN"]
env_vars = ["ZABBIX_HOST", "NETBOX_HOST", "NETBOX_TOKEN"]
if "ZABBIX_TOKEN" in environ:
env_vars.append("ZABBIX_TOKEN")
else:
env_vars.append("ZABBIX_USER")
env_vars.append("ZABBIX_PASS")
for var in env_vars:
if var not in environ:
e = f"Environment variable {var} has not been defined."
logger.error(e)
raise EnvironmentVarError(e)
# Get all virtual environment variables
if "ZABBIX_TOKEN" in env_vars:
zabbix_user = None
zabbix_pass = None
zabbix_token = environ.get("ZABBIX_TOKEN")
else:
zabbix_user = environ.get("ZABBIX_USER")
zabbix_pass = environ.get("ZABBIX_PASS")
zabbix_token = None
zabbix_host = environ.get("ZABBIX_HOST")
zabbix_user = environ.get("ZABBIX_USER")
zabbix_pass = environ.get("ZABBIX_PASS")
netbox_host = environ.get("NETBOX_HOST")
netbox_token = environ.get("NETBOX_TOKEN")
# Set Netbox API
netbox = api(netbox_host, token=netbox_token, threading=True)
# Check if the provided Hostgroup layout is valid
hg_objects = hostgroup_format.split("/")
allowed_objects = ["dev_location", "dev_role", "manufacturer", "region",
allowed_objects = ["location", "role", "manufacturer", "region",
"site", "site_group", "tenant", "tenant_group"]
# Create API call to get all custom fields which are on the device objects
device_cfs = netbox.extras.custom_fields.filter(type="text", content_type_id=23)
try:
device_cfs = list(netbox.extras.custom_fields.filter(type="text", content_type_id=23))
except RequestsConnectionError:
logger.error(f"Unable to connect to Netbox with URL {netbox_host}."
" Please check the URL and status of Netbox.")
sys.exit(1)
except NBRequestError as e:
logger.error(f"Netbox error: {e}")
sys.exit(1)
for cf in device_cfs:
allowed_objects.append(cf.name)
for object in hg_objects:
if(object not in allowed_objects):
e = (f"Hostgroup item {object} is not valid. Make sure you"
for hg_object in hg_objects:
if hg_object not in allowed_objects:
e = (f"Hostgroup item {hg_object} is not valid. Make sure you"
" use valid items and seperate them with '/'.")
logger.error(e)
raise HostgroupError(e)
# Set Zabbix API
try:
zabbix = ZabbixAPI(zabbix_host)
zabbix.login(zabbix_user, zabbix_pass)
except ZabbixAPIException as e:
e = f"Zabbix returned the following error: {str(e)}."
if not zabbix_token:
zabbix = ZabbixAPI(zabbix_host, user=zabbix_user, password=zabbix_pass)
else:
zabbix = ZabbixAPI(zabbix_host, token=zabbix_token)
zabbix.check_auth()
except (APIRequestError, ProcessingError) as e:
e = f"Zabbix returned the following error: {str(e)}"
logger.error(e)
sys.exit(1)
# Set API parameter mapping based on API version
if not str(zabbix.version).startswith('7'):
proxy_name = "host"
else:
proxy_name = "name"
# Get all Zabbix and Netbox data
netbox_devices = netbox.dcim.devices.filter(**nb_device_filter)
netbox_devices = list(netbox.dcim.devices.filter(**nb_device_filter))
netbox_vms = []
if sync_vms:
netbox_vms = list(netbox.virtualization.virtual_machines.filter(**nb_vm_filter))
netbox_site_groups = convert_recordset((netbox.dcim.site_groups.all()))
netbox_regions = convert_recordset(netbox.dcim.regions.all())
netbox_journals = netbox.extras.journal_entries
zabbix_groups = zabbix.hostgroup.get(output=['groupid', 'name'])
zabbix_templates = zabbix.template.get(output=['templateid', 'name'])
zabbix_proxys = zabbix.proxy.get(output=['proxyid', 'host'])
zabbix_proxies = zabbix.proxy.get(output=['proxyid', proxy_name])
# Set empty list for proxy processing Zabbix <= 6
zabbix_proxygroups = []
if str(zabbix.version).startswith('7'):
zabbix_proxygroups = zabbix.proxygroup.get(output=["proxy_groupid", "name"])
# Sanitize proxy data
if proxy_name == "host":
for proxy in zabbix_proxies:
proxy['name'] = proxy.pop('host')
# Prepare list of all proxy and proxy_groups
zabbix_proxy_list = proxy_prepper(zabbix_proxies, zabbix_proxygroups)
# Get Netbox API version
nb_version = netbox.version
# Go through all Netbox devices
for nb_vm in netbox_vms:
try:
vm = VirtualMachine(nb_vm, zabbix, netbox_journals, nb_version,
create_journal, logger)
logger.debug(f"Host {vm.name}: started operations on VM.")
vm.set_vm_template()
# Check if a valid template has been found for this VM.
if not vm.zbx_template_names:
continue
vm.set_hostgroup(vm_hostgroup_format,netbox_site_groups,netbox_regions)
# Check if a valid hostgroup has been found for this VM.
if not vm.hostgroup:
continue
# Temporary disable inventory sync for VM's
# vm.set_inventory(nb_vm)
# Checks if device is in cleanup state
if vm.status in zabbix_device_removal:
if vm.zabbix_id:
# Delete device from Zabbix
# and remove hostID from Netbox.
vm.cleanup()
logger.info(f"VM {vm.name}: cleanup complete")
continue
# Device has been added to Netbox
# but is not in Activate state
logger.info(f"VM {vm.name}: skipping since this VM is "
f"not in the active state.")
continue
# Check if the VM is in the disabled state
if vm.status in zabbix_device_disable:
vm.zabbix_state = 1
# Check if VM is already in Zabbix
if vm.zabbix_id:
vm.ConsistencyCheck(zabbix_groups, zabbix_templates,
zabbix_proxy_list, full_proxy_sync,
create_hostgroups)
continue
# Add hostgroup is config is set
if create_hostgroups:
# Create new hostgroup. Potentially multiple groups if nested
hostgroups = vm.createZabbixHostgroup(zabbix_groups)
# go through all newly created hostgroups
for group in hostgroups:
# Add new hostgroups to zabbix group list
zabbix_groups.append(group)
# Add VM to Zabbix
vm.createInZabbix(zabbix_groups, zabbix_templates,
zabbix_proxy_list)
except SyncError:
pass
for nb_device in netbox_devices:
try:
device = NetworkDevice(nb_device, zabbix, netbox_journals,
create_journal)
device.set_hostgroup(hostgroup_format)
# Set device instance set data such as hostgroup and template information.
device = PhysicalDevice(nb_device, zabbix, netbox_journals, nb_version,
create_journal, logger)
logger.debug(f"Host {device.name}: started operations on device.")
device.set_template(templates_config_context, templates_config_context_overrule)
# Check if a valid template has been found for this VM.
if not device.zbx_template_names:
continue
device.set_hostgroup(hostgroup_format,netbox_site_groups,netbox_regions)
# Check if a valid hostgroup has been found for this VM.
if not device.hostgroup:
continue
device.set_inventory(nb_device)
# Checks if device is part of cluster.
# Requires clustering variable
if(device.isCluster() and clustering):
# Check if device is master or slave
if(device.promoteMasterDevice()):
e = (f"Device {device.name} is "
if device.isCluster() and clustering:
# Check if device is primary or secondary
if device.promoteMasterDevice():
e = (f"Device {device.name}: is "
f"part of cluster and primary.")
logger.info(e)
else:
# Device is secondary in cluster.
# Don't continue with this device.
e = (f"Device {device.name} is part of cluster "
e = (f"Device {device.name}: is part of cluster "
f"but not primary. Skipping this host...")
logger.info(e)
continue
# Checks if device is in cleanup state
if(device.status in zabbix_device_removal):
if(device.zabbix_id):
if device.status in zabbix_device_removal:
if device.zabbix_id:
# Delete device from Zabbix
# and remove hostID from Netbox.
device.cleanup()
logger.info(f"Cleaned up host {device.name}.")
else:
# Device has been added to Netbox
# but is not in Activate state
logger.info(f"Skipping host {device.name} since its "
f"not in the active state.")
logger.info(f"Device {device.name}: cleanup complete")
continue
# Device has been added to Netbox
# but is not in Activate state
logger.info(f"Device {device.name}: skipping since this device is "
f"not in the active state.")
continue
elif(device.status in zabbix_device_disable):
# Check if the device is in the disabled state
if device.status in zabbix_device_disable:
device.zabbix_state = 1
# Add hostgroup is variable is True
# and Hostgroup is not present in Zabbix
if(create_hostgroups):
for group in zabbix_groups:
# If hostgroup is already present in Zabbix
if(group["name"] == device.hostgroup):
break
else:
# Create new hostgroup
hostgroup = device.createZabbixHostgroup()
zabbix_groups.append(hostgroup)
# Device is already present in Zabbix
if(device.zabbix_id):
# Check if device is already in Zabbix
if device.zabbix_id:
device.ConsistencyCheck(zabbix_groups, zabbix_templates,
zabbix_proxys, full_proxy_sync)
zabbix_proxy_list, full_proxy_sync,
create_hostgroups)
continue
# Add hostgroup is config is set
if create_hostgroups:
# Create new hostgroup. Potentially multiple groups if nested
hostgroups = device.createZabbixHostgroup(zabbix_groups)
# go through all newly created hostgroups
for group in hostgroups:
# Add new hostgroups to zabbix group list
zabbix_groups.append(group)
# Add device to Zabbix
else:
device.createInZabbix(zabbix_groups, zabbix_templates,
zabbix_proxys)
device.createInZabbix(zabbix_groups, zabbix_templates,
zabbix_proxy_list)
except SyncError:
pass
class SyncError(Exception):
pass
class SyncExternalError(SyncError):
pass
class SyncInventoryError(SyncError):
pass
class SyncDuplicateError(SyncError):
pass
class EnvironmentVarError(SyncError):
pass
class InterfaceConfigError(SyncError):
pass
class ProxyConfigError(SyncError):
pass
class HostgroupError(SyncError):
pass
class TemplateError(SyncError):
pass
class NetworkDevice():
"""
Represents Network device.
INPUT: (Netbox device class, ZabbixAPI class, journal flag, NB journal class)
"""
def __init__(self, nb, zabbix, nb_journal_class, journal=None):
self.nb = nb
self.id = nb.id
self.name = nb.name
self.status = nb.status.label
self.zabbix = zabbix
self.tenant = nb.tenant
self.config_context = nb.config_context
self.zbxproxy = "0"
self.zabbix_state = 0
self.journal = journal
self.nb_journals = nb_journal_class
self._setBasics()
def _setBasics(self):
"""
Sets basic information like IP address.
"""
# Return error if device does not have primary IP.
if(self.nb.primary_ip):
self.cidr = self.nb.primary_ip.address
self.ip = self.cidr.split("/")[0]
else:
e = f"Device {self.name}: no primary IP."
logger.warning(e)
raise SyncInventoryError(e)
# Check if device has custom field for ZBX ID
if(device_cf in self.nb.custom_fields):
self.zabbix_id = self.nb.custom_fields[device_cf]
else:
e = f"Custom field {device_cf} not found for {self.name}."
logger.warning(e)
raise SyncInventoryError(e)
def set_hostgroup(self, format):
"""Set the hostgroup for this device"""
# Get all variables from the NB data
dev_location = str(self.nb.location) if self.nb.location else None
dev_role = self.nb.device_role.name
manufacturer = self.nb.device_type.manufacturer.name
region = str(self.nb.site.region) if self.nb.site.region else None
site = self.nb.site.name
site_group = str(self.nb.site.group) if self.nb.site.group else None
tenant = str(self.tenant) if self.tenant else None
tenant_group = str(self.tenant.group) if tenant else None
# Set mapper for string -> variable
hostgroup_vars = {"dev_location": dev_location, "dev_role": dev_role,
"manufacturer": manufacturer, "region": region,
"site": site, "site_group": site_group,
"tenant": tenant, "tenant_group": tenant_group}
# Generate list based off string input format
hg_items = format.split("/")
hostgroup = ""
# Go through all hostgroup items
for item in hg_items:
# Check if the variable (such as Tenant) is empty.
if(not hostgroup_vars[item]):
continue
# Check if the item is a custom field name
if(item not in hostgroup_vars):
cf_value = self.nb.custom_fields[item] if item in self.nb.custom_fields else None
if(cf_value):
# If there is a cf match, add the value of this cf to the hostgroup
hostgroup += cf_value + "/"
# Should there not be a match, this means that
# the variable is invalid. Skip regardless.
continue
# Add value of predefined variable to hostgroup format
hostgroup += hostgroup_vars[item] + "/"
# If the final hostgroup variable is empty
if(not hostgroup):
e = (f"{self.name} has no reliable hostgroup. This is"
"most likely due to the use of custom fields that are empty.")
logger.error(e)
raise SyncInventoryError(e)
# Remove final inserted "/" and set hostgroup to class var
self.hostgroup = hostgroup.rstrip("/")
def set_template(self, prefer_config_context, overrule_custom):
self.zbx_template_names = None
# Gather templates ONLY from the device specific context
if prefer_config_context:
try:
self.zbx_template_names = self.get_templates_context()
except TemplateError as e:
logger.warning(e)
return True
# Gather templates from the custom field but overrule
# them should there be any device specific templates
if overrule_custom:
try:
self.zbx_template_names = self.get_templates_context()
except TemplateError:
pass
if not self.zbx_template_names:
self.zbx_template_names = self.get_templates_cf()
return True
# Gather templates ONLY from the custom field
self.zbx_template_names = self.get_templates_cf()
return True
def get_templates_cf(self):
# Get Zabbix templates from the device type
device_type_cfs = self.nb.device_type.custom_fields
# Check if the ZBX Template CF is present
if(template_cf in device_type_cfs):
# Set value to template
return [device_type_cfs[template_cf]]
else:
# Custom field not found, return error
e = (f"Custom field {template_cf} not "
f"found for {self.nb.device_type.manufacturer.name}"
f" - {self.nb.device_type.display}.")
raise TemplateError(e)
def get_templates_context(self):
# Get Zabbix templates from the device context
if("zabbix" not in self.config_context):
e = ("Key 'zabbix' not found in config "
f"context for template host {self.name}")
raise TemplateError(e)
if("templates" not in self.config_context["zabbix"]):
e = ("Key 'zabbix' not found in config "
f"context for template host {self.name}")
raise TemplateError(e)
return self.config_context["zabbix"]["templates"]
def isCluster(self):
"""
Checks if device is part of cluster.
"""
if(self.nb.virtual_chassis):
return True
else:
return False
def getClusterMaster(self):
"""
Returns chassis master ID.
"""
if(not self.isCluster()):
e = (f"Unable to proces {self.name} for cluster calculation: "
f"not part of a cluster.")
logger.warning(e)
raise SyncInventoryError(e)
elif(not self.nb.virtual_chassis.master):
e = (f"{self.name} is part of a Netbox virtual chassis which does "
"not have a master configured. Skipping for this reason.")
logger.error(e)
raise SyncInventoryError(e)
else:
return self.nb.virtual_chassis.master.id
def promoteMasterDevice(self):
"""
If device is Primary in cluster,
promote device name to the cluster name.
Returns True if succesfull, returns False if device is secondary.
"""
masterid = self.getClusterMaster()
if(masterid == self.id):
logger.debug(f"Device {self.name} is primary cluster member. "
f"Modifying hostname from {self.name} to " +
f"{self.nb.virtual_chassis.name}.")
self.name = self.nb.virtual_chassis.name
return True
else:
logger.debug(f"Device {self.name} is non-primary cluster member.")
return False
def zbxTemplatePrepper(self, templates):
"""
Returns Zabbix template IDs
INPUT: list of templates from Zabbix
OUTPUT: True
"""
# Check if there are templates defined
if(not self.zbx_template_names):
e = (f"No templates found for device {self.name}")
logger.info(e)
raise SyncInventoryError()
# Set variable to empty list
self.zbx_templates = []
# Go through all templates definded in Netbox
for nb_template in self.zbx_template_names:
template_match = False
# Go through all templates found in Zabbix
for zbx_template in templates:
# If the template names match
if(zbx_template['name'] == nb_template):
# Set match variable to true, add template details
# to class variable and return debug log
template_match = True
self.zbx_templates.append({"templateid": zbx_template['templateid'],
"name": zbx_template['name']})
e = (f"Found template {zbx_template['name']}"
f" for host {self.name}.")
logger.debug(e)
# Return error should the template not be found in Zabbix
if(not template_match):
e = (f"Unable to find template {nb_template} "
f"for host {self.name} in Zabbix. Skipping host...")
logger.warning(e)
raise SyncInventoryError(e)
def getZabbixGroup(self, groups):
"""
Returns Zabbix group ID
INPUT: list of hostgroups
OUTPUT: True / False
"""
# Go through all groups
for group in groups:
if(group['name'] == self.hostgroup):
self.group_id = group['groupid']
e = (f"Found group {group['name']} for host {self.name}.")
logger.debug(e)
return True
else:
e = (f"Unable to find group '{self.hostgroup}' "
f"for host {self.name} in Zabbix.")
logger.warning(e)
raise SyncInventoryError(e)
def cleanup(self):
"""
Removes device from external resources.
Resets custom fields in Netbox.
"""
if(self.zabbix_id):
try:
self.zabbix.host.delete(self.zabbix_id)
self.nb.custom_fields[device_cf] = None
self.nb.save()
e = f"Deleted host {self.name} from Zabbix."
logger.info(e)
self.create_journal_entry("warning", "Deleted host from Zabbix")
except ZabbixAPIException as e:
e = f"Zabbix returned the following error: {str(e)}."
logger.error(e)
raise SyncExternalError(e)
def _zabbixHostnameExists(self):
"""
Checks if hostname exists in Zabbix.
"""
host = self.zabbix.host.get(filter={'name': self.name}, output=[])
if(host):
return True
else:
return False
def setInterfaceDetails(self):
"""
Checks interface parameters from Netbox and
creates a model for the interface to be used in Zabbix.
"""
try:
# Initiate interface class
interface = ZabbixInterface(self.nb.config_context, self.ip)
# Check if Netbox has device context.
# If not fall back to old config.
if(interface.get_context()):
# If device is SNMP type, add aditional information.
if(interface.interface["type"] == 2):
interface.set_snmp()
else:
interface.set_default()
return [interface.interface]
except InterfaceConfigError as e:
e = f"{self.name}: {e}"
logger.warning(e)
raise SyncInventoryError(e)
def setProxy(self, proxy_list):
# check if Zabbix Proxy has been defined in config context
if("zabbix" in self.nb.config_context):
if("proxy" in self.nb.config_context["zabbix"]):
proxy = self.nb.config_context["zabbix"]["proxy"]
# Try matching proxy
for px in proxy_list:
if(px["host"] == proxy):
self.zbxproxy = px["proxyid"]
logger.debug(f"Found proxy {proxy}"
f" for {self.name}.")
return True
else:
e = f"{self.name}: Defined proxy {proxy} not found."
logger.warning(e)
return False
def createInZabbix(self, groups, templates, proxys,
description="Host added by Netbox sync script."):
"""
Creates Zabbix host object with parameters from Netbox object.
"""
# Check if hostname is already present in Zabbix
if(not self._zabbixHostnameExists()):
# Get group and template ID's for host
if(not self.getZabbixGroup(groups)):
raise SyncInventoryError()
self.zbxTemplatePrepper(templates)
# Set interface, group and template configuration
interfaces = self.setInterfaceDetails()
groups = [{"groupid": self.group_id}]
# Set Zabbix proxy if defined
self.setProxy(proxys)
# Add host to Zabbix
try:
host = self.zabbix.host.create(host=self.name,
status=self.zabbix_state,
interfaces=interfaces,
groups=groups,
templates=self.zbx_templates,
proxy_hostid=self.zbxproxy,
description=description)
self.zabbix_id = host["hostids"][0]
except ZabbixAPIException as e:
e = f"Couldn't add {self.name}, Zabbix returned {str(e)}."
logger.error(e)
raise SyncExternalError(e)
# Set Netbox custom field to hostID value.
self.nb.custom_fields[device_cf] = int(self.zabbix_id)
self.nb.save()
msg = f"Created host {self.name} in Zabbix."
logger.info(msg)
self.create_journal_entry("success", msg)
else:
e = f"Unable to add {self.name} to Zabbix: host already present."
logger.warning(e)
def createZabbixHostgroup(self):
"""
Creates Zabbix host group based on hostgroup format.
"""
try:
groupid = self.zabbix.hostgroup.create(name=self.hostgroup)
e = f"Added hostgroup '{self.hostgroup}'."
logger.info(e)
data = {'groupid': groupid["groupids"][0], 'name': self.hostgroup}
return data
except ZabbixAPIException as e:
e = f"Couldn't add hostgroup, Zabbix returned {str(e)}."
logger.error(e)
raise SyncExternalError(e)
def updateZabbixHost(self, **kwargs):
"""
Updates Zabbix host with given parameters.
INPUT: Key word arguments for Zabbix host object.
"""
try:
self.zabbix.host.update(hostid=self.zabbix_id, **kwargs)
except ZabbixAPIException as e:
e = f"Zabbix returned the following error: {str(e)}."
logger.error(e)
raise SyncExternalError(e)
logger.info(f"Updated host {self.name} with data {kwargs}.")
self.create_journal_entry("info", f"Updated host in Zabbix with latest NB data.")
def ConsistencyCheck(self, groups, templates, proxys, proxy_power):
"""
Checks if Zabbix object is still valid with Netbox parameters.
"""
self.getZabbixGroup(groups)
self.zbxTemplatePrepper(templates)
self.setProxy(proxys)
host = self.zabbix.host.get(filter={'hostid': self.zabbix_id},
selectInterfaces=['type', 'ip',
'port', 'details',
'interfaceid'],
selectGroups=["groupid"],
selectParentTemplates=["templateid"])
if(len(host) > 1):
e = (f"Got {len(host)} results for Zabbix hosts "
f"with ID {self.zabbix_id} - hostname {self.name}.")
logger.error(e)
raise SyncInventoryError(e)
elif(len(host) == 0):
e = (f"No Zabbix host found for {self.name}. "
f"This is likely the result of a deleted Zabbix host "
f"without zeroing the ID field in Netbox.")
logger.error(e)
raise SyncInventoryError(e)
else:
host = host[0]
if(host["host"] == self.name):
logger.debug(f"Device {self.name}: hostname in-sync.")
else:
logger.warning(f"Device {self.name}: hostname OUT of sync. "
f"Received value: {host['host']}")
self.updateZabbixHost(host=self.name)
# Check if the templates are in-sync
if(not self.zbx_template_comparer(host["parentTemplates"])):
logger.warning(f"Device {self.name}: template(s) OUT of sync.")
# Update Zabbix with NB templates and clear any old / lost templates
self.updateZabbixHost(templates_clear=host["parentTemplates"], templates=self.zbx_templates)
else:
logger.debug(f"Device {self.name}: template(s) in-sync.")
for group in host["groups"]:
if(group["groupid"] == self.group_id):
logger.debug(f"Device {self.name}: hostgroup in-sync.")
break
else:
logger.warning(f"Device {self.name}: hostgroup OUT of sync.")
self.updateZabbixHost(groups={'groupid': self.group_id})
if(int(host["status"]) == self.zabbix_state):
logger.debug(f"Device {self.name}: status in-sync.")
else:
logger.warning(f"Device {self.name}: status OUT of sync.")
self.updateZabbixHost(status=str(self.zabbix_state))
# Check if a proxy has been defined
if(self.zbxproxy != "0"):
# Check if expected proxyID matches with configured proxy
if(host["proxy_hostid"] == self.zbxproxy):
logger.debug(f"Device {self.name}: proxy in-sync.")
else:
# Proxy diff, update value
logger.warning(f"Device {self.name}: proxy OUT of sync.")
self.updateZabbixHost(proxy_hostid=self.zbxproxy)
else:
if(not host["proxy_hostid"] == "0"):
if(proxy_power):
# Variable full_proxy_sync has been enabled
# delete the proxy link in Zabbix
self.updateZabbixHost(proxy_hostid=self.zbxproxy)
else:
# Instead of deleting the proxy config in zabbix and
# forcing potential data loss,
# an error message is displayed.
logger.error(f"Device {self.name} is configured "
f"with proxy in Zabbix but not in Netbox. The"
" -p flag was ommited: no "
"changes have been made.")
# If only 1 interface has been found
if(len(host['interfaces']) == 1):
updates = {}
# Go through each key / item and check if it matches Zabbix
for key, item in self.setInterfaceDetails()[0].items():
# Check if Netbox value is found in Zabbix
if(key in host["interfaces"][0]):
# If SNMP is used, go through nested dict
# to compare SNMP parameters
if(type(item) == dict and key == "details"):
for k, i in item.items():
if(k in host["interfaces"][0][key]):
# Set update if values don't match
if(host["interfaces"][0][key][k] != str(i)):
# If dict has not been created, add it
if(key not in updates):
updates[key] = {}
updates[key][k] = str(i)
# If SNMP version has been changed
# break loop and force full SNMP update
if(k == "version"):
break
# Force full SNMP config update
# when version has changed.
if(key in updates):
if("version" in updates[key]):
for k, i in item.items():
updates[key][k] = str(i)
continue
# Set update if values don't match
if(host["interfaces"][0][key] != str(item)):
updates[key] = item
if(updates):
# If interface updates have been found: push to Zabbix
logger.warning(f"Device {self.name}: Interface OUT of sync.")
if("type" in updates):
# Changing interface type not supported. Raise exception.
e = (f"Device {self.name}: changing interface type to "
f"{str(updates['type'])} is not supported.")
logger.error(e)
raise InterfaceConfigError(e)
# Set interfaceID for Zabbix config
updates["interfaceid"] = host["interfaces"][0]['interfaceid']
try:
# API call to Zabbix
self.zabbix.hostinterface.update(updates)
e = f"Solved {self.name} interface conflict."
logger.info(e)
self.create_journal_entry("info", e)
except ZabbixAPIException as e:
e = f"Zabbix returned the following error: {str(e)}."
logger.error(e)
raise SyncExternalError(e)
else:
# If no updates are found, Zabbix interface is in-sync
e = f"Device {self.name}: interface in-sync."
logger.debug(e)
else:
e = (f"Device {self.name} has unsupported interface configuration."
f" Host has total of {len(host['interfaces'])} interfaces. "
"Manual interfention required.")
logger.error(e)
SyncInventoryError(e)
def create_journal_entry(self, severity, message):
# Send a new Journal entry to Netbox. Usefull for viewing actions
# in Netbox without having to look in Zabbix or the script log output
if(self.journal):
# Check if the severity is valid
if severity not in ["info", "success", "warning", "danger"]:
logger.warning(f"Value {severity} not valid for NB journal entries.")
return False
journal = {"assigned_object_type": "dcim.device",
"assigned_object_id": self.id,
"kind": severity,
"comments": message
}
try:
self.nb_journals.create(journal)
return True
logger.debug(f"Crated journal entry in NB for host {self.name}")
except pynetbox.RequestError as e:
logger.warning("Unable to create journal entry for "
f"{self.name}: NB returned {e}")
def zbx_template_comparer(self, tmpls_from_zabbix):
"""
Compares the Netbox and Zabbix templates with each other.
Should there be a mismatch then the function will return false
INPUT: list of NB and ZBX templates
OUTPUT: Boolean True/False
"""
succesfull_templates = []
# Go through each Netbox template
for nb_tmpl in self.zbx_templates:
# Go through each Zabbix template
for pos, zbx_tmpl in enumerate(tmpls_from_zabbix):
# Check if template IDs match
if(nb_tmpl["templateid"] == zbx_tmpl["templateid"]):
# Templates match. Remove this template from the Zabbix templates
# and add this NB template to the list of successfull templates
tmpls_from_zabbix.pop(pos)
succesfull_templates.append(nb_tmpl)
logger.debug(f"Device {self.name}: template {nb_tmpl['name']} is present in Zabbix.")
break
if(len(succesfull_templates) == len(self.zbx_templates) and
len(tmpls_from_zabbix) == 0):
# All of the Netbox templates have been confirmed as successfull
# and the ZBX template list is empty. This means that
# all of the templates match.
return True
return False
class ZabbixInterface():
"""Class that represents a Zabbix interface."""
def __init__(self, context, ip):
self.context = context
self.ip = ip
self.skelet = {"main": "1", "useip": "1", "dns": "", "ip": self.ip}
self.interface = self.skelet
def get_context(self):
# check if Netbox custom context has been defined.
if("zabbix" in self.context):
zabbix = self.context["zabbix"]
if("interface_type" in zabbix and "interface_port" in zabbix):
self.interface["type"] = zabbix["interface_type"]
self.interface["port"] = zabbix["interface_port"]
return True
else:
return False
else:
return False
def set_snmp(self):
# Check if interface is type SNMP
if(self.interface["type"] == 2):
# Checks if SNMP settings are defined in Netbox
if("snmp" in self.context["zabbix"]):
snmp = self.context["zabbix"]["snmp"]
self.interface["details"] = {}
# Checks if bulk config has been defined
if("bulk" in snmp):
self.interface["details"]["bulk"] = str(snmp.pop("bulk"))
else:
# Fallback to bulk enabled if not specified
self.interface["details"]["bulk"] = "1"
# SNMP Version config is required in Netbox config context
if(snmp.get("version")):
self.interface["details"]["version"] = str(snmp.pop("version"))
else:
e = "SNMP version option is not defined."
raise InterfaceConfigError(e)
# If version 1 or 2 is used, get community string
if(self.interface["details"]["version"] in ['1','2']):
if("community" in snmp):
# Set SNMP community to confix context value
community = snmp["community"]
else:
# Set SNMP community to default
community = "{$SNMP_COMMUNITY}"
self.interface["details"]["community"] = str(community)
# If version 3 has been used, get all
# SNMPv3 Netbox related configs
elif(self.interface["details"]["version"] == '3'):
items = ["securityname", "securitylevel", "authpassphrase",
"privpassphrase", "authprotocol", "privprotocol",
"contextname"]
for key, item in snmp.items():
if(key in items):
self.interface["details"][key] = str(item)
else:
e = "Unsupported SNMP version."
raise InterfaceConfigError(e)
else:
e = "Interface type SNMP but no parameters provided."
raise InterfaceConfigError(e)
else:
e = "Interface type is not SNMP, unable to set SNMP details"
raise InterfaceConfigError(e)
def set_default(self):
# Set default config to SNMPv2,port 161 and community macro.
self.interface = self.skelet
self.interface["type"] = "2"
self.interface["port"] = "161"
self.interface["details"] = {"version": "2",
"community": "{$SNMP_COMMUNITY}",
"bulk": "1"}
if(__name__ == "__main__"):
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description='A script to sync Zabbix with Netbox device data.'
)

View File

@@ -1,2 +1,2 @@
pynetbox
pyzabbix
zabbix_utils