Merge branch 'feature' into 9608-drf-spectacular2

.github/ISSUE_TEMPLATE/bug_report.yaml

@@ -14,7 +14,7 @@ body:
     attributes:
       label: NetBox version
       description: What version of NetBox are you currently running?
-      placeholder: v3.4.6
+      placeholder: v3.4.7
     validations:
       required: true
   - type: dropdown

.github/ISSUE_TEMPLATE/feature_request.yaml

@@ -14,7 +14,7 @@ body:
     attributes:
       label: NetBox version
       description: What version of NetBox are you currently running?
-      placeholder: v3.4.6
+      placeholder: v3.4.7
     validations:
       required: true
   - type: dropdown

base_requirements.txt

@@ -129,7 +129,8 @@ social-auth-core
 
 # Django app for social-auth-core
 # https://github.com/python-social-auth/social-app-django
-social-auth-app-django
+# See https://github.com/python-social-auth/social-app-django/issues/429
+social-auth-app-django==5.0.0
 
 # SVG image rendering (used for rack elevations)
 # https://github.com/mozman/svgwrite

docs/administration/housekeeping.md

@@ -4,7 +4,7 @@ NetBox includes a `housekeeping` management command that should be run nightly.
 
 * Clearing expired authentication sessions from the database
 * Deleting changelog records older than the configured [retention time](../configuration/miscellaneous.md#changelog_retention)
-* Deleting job result records older than the configured [retention time](../configuration/miscellaneous.md#jobresult_retention)
+* Deleting job result records older than the configured [retention time](../configuration/miscellaneous.md#job_retention)
 * Check for new NetBox releases (if [`RELEASE_CHECK_URL`](../configuration/miscellaneous.md#release_check_url) is set)
 
 This command can be invoked directly, or by using the shell script provided at `/opt/netbox/contrib/netbox-housekeeping.sh`. This script can be linked from your cron scheduler's daily jobs directory (e.g. `/etc/cron.daily`) or referenced directly within the cron configuration file.

docs/configuration/index.md

@@ -26,7 +26,7 @@ Some configuration parameters are primarily controlled via NetBox's admin interf
 * [`DEFAULT_USER_PREFERENCES`](./default-values.md#default_user_preferences)
 * [`ENFORCE_GLOBAL_UNIQUE`](./miscellaneous.md#enforce_global_unique)
 * [`GRAPHQL_ENABLED`](./miscellaneous.md#graphql_enabled)
-* [`JOBRESULT_RETENTION`](./miscellaneous.md#jobresult_retention)
+* [`JOB_RETENTION`](./miscellaneous.md#job_retention)
 * [`MAINTENANCE_MODE`](./miscellaneous.md#maintenance_mode)
 * [`MAPS_URL`](./miscellaneous.md#maps_url)
 * [`MAX_PAGE_SIZE`](./miscellaneous.md#max_page_size)

docs/configuration/miscellaneous.md

@@ -87,14 +87,16 @@ Setting this to False will disable the GraphQL API.
 
 ---
 
-## JOBRESULT_RETENTION
+## JOB_RETENTION
 
 !!! tip "Dynamic Configuration Parameter"
 
+!!! note
+    This parameter was renamed from `JOBRESULT_RETENTION` in NetBox v3.5.
+
 Default: 90
 
-The number of days to retain job results (scripts and reports). Set this to `0` to retain
-job results in the database indefinitely.
+The number of days to retain job results (scripts and reports). Set this to `0` to retain job results in the database indefinitely.
 
 !!! warning
     If enabling indefinite job results retention, it is recommended to periodically delete old entries. Otherwise, the database may eventually exceed capacity.

docs/configuration/remote-authentication.md

@@ -16,7 +16,7 @@ If true, NetBox will automatically create local accounts for users authenticated
 
 Default: `'netbox.authentication.RemoteUserBackend'`
 
-This is the Python path to the custom [Django authentication backend](https://docs.djangoproject.com/en/stable/topics/auth/customizing/) to use for external user authentication. NetBox provides two built-in backends (listed below), though custom authentication backends may also be provided by other packages or plugins.
+This is the Python path to the custom [Django authentication backend](https://docs.djangoproject.com/en/stable/topics/auth/customizing/) to use for external user authentication. NetBox provides two built-in backends (listed below), though custom authentication backends may also be provided by other packages or plugins. Provide a string for a single backend, or an iterable for multiple backends, which will be attempted in the order given.
 
 * `netbox.authentication.RemoteUserBackend`
 * `netbox.authentication.LDAPBackend`

docs/configuration/system.md

@@ -38,7 +38,7 @@ In order to send email, NetBox needs an email server configured. The following i
 * `SERVER` - Hostname or IP address of the email server (use `localhost` if running locally)
 * `PORT` - TCP port to use for the connection (default: `25`)
 * `USERNAME` - Username with which to authenticate
-* `PASSSWORD` - Password with which to authenticate
+* `PASSWORD` - Password with which to authenticate
 * `USE_SSL` - Use SSL when connecting to the server (default: `False`)
 * `USE_TLS` - Use TLS when connecting to the server (default: `False`)
 * `SSL_CERTFILE` - Path to the PEM-formatted SSL certificate file (optional)
@@ -73,6 +73,14 @@ Determines if localization features are enabled or not. This should only be enab
 
 ---
 
+## GIT_PATH
+
+Default: `git`
+
+The system path to the `git` executable, used by the synchronization backend for remote git repositories.
+
+---
+
 ## HTTP_PROXIES
 
 Default: None

docs/customization/custom-scripts.md

@@ -79,7 +79,22 @@ A human-friendly description of what your script does.
 
 ### `field_order`
 
-By default, script variables will be ordered in the form as they are defined in the script. `field_order` may be defined as an iterable of field names to determine the order in which variables are rendered. Any fields not included in this iterable be listed last.
+By default, script variables will be ordered in the form as they are defined in the script. `field_order` may be defined as an iterable of field names to determine the order in which variables are rendered within a default "Script Data" group. Any fields not included in this iterable be listed last. If `fieldsets` is defined, `field_order` will be ignored.  A fieldset group for "Script Execution Parameters" will be added to the end of the form by default for the user.
+
+### `fieldsets`
+
+`fieldsets` may be defined as an iterable of field groups and their field names to determine the order in which variables are group and rendered. Any fields not included in this iterable will not be displayed in the form. If `fieldsets` is defined, `field_order` will be ignored.  A fieldset group for "Script Execution Parameters" will be added to the end of the fieldsets by default for the user.
+
+An example fieldset definition is provided below:
+
+```python
+class MyScript(Script):
+    class Meta:
+        fieldsets = (
+            ('First group', ('field1', 'field2', 'field3')),
+            ('Second group', ('field4', 'field5')),
+        )
+```
 
 ### `commit_default`
 

docs/development/models.md

@@ -18,7 +18,7 @@ Depending on its classification, each NetBox model may support various features
 | [Custom links](../customization/custom-links.md)           | `CustomLinksMixin`      | `custom_links`     | These models support the assignment of custom links                            |
 | [Custom validation](../customization/custom-validation.md) | `CustomValidationMixin` | -                  | Supports the enforcement of custom validation rules                            |
 | [Export templates](../customization/export-templates.md)   | `ExportTemplatesMixin`  | `export_templates` | Users can create custom export templates for these models                      |
-| [Job results](../features/background-jobs.md)              | `JobResultsMixin`       | `job_results`      | Users can create custom export templates for these models                      |
+| [Job results](../features/background-jobs.md)              | `JobsMixin`             | `jobs`             | Users can create custom export templates for these models                      |
 | [Journaling](../features/journaling.md)                    | `JournalingMixin`       | `journaling`       | These models support persistent historical commentary                          |
 | [Synchronized data](../integrations/synchronized-data.md)  | `SyncedDataMixin`       | `synced_data`      | Certain model data can be automatically synchronized from a remote data source |
 | [Tagging](../models/extras/tag.md)                         | `TagsMixin`             | `tags`             | The models can be tagged with user-defined tags                                |
@@ -32,6 +32,7 @@ These are considered the "core" application models which are used to model netwo
 
 * [circuits.Circuit](../models/circuits/circuit.md)
 * [circuits.Provider](../models/circuits/provider.md)
+* [circuits.ProviderAccount](../models/circuits/provideracount.md)
 * [circuits.ProviderNetwork](../models/circuits/providernetwork.md)
 * [core.DataSource](../models/core/datasource.md)
 * [dcim.Cable](../models/dcim/cable.md)

docs/development/search.md

@@ -29,7 +29,7 @@ A SearchIndex subclass defines both its model and a list of two-tuples specifyin
 | 60     | Unique serialized attribute (per related object) | Device.serial                                      |
 | 100    | Primary human identifier                         | Device.name, Circuit.cid, Cable.label              |
 | 110    | Slug                                             | Site.slug                                          |
-| 200    | Secondary identifier                             | Provider.account, DeviceType.part_number           |
+| 200    | Secondary identifier                             | ProviderAccount.account, DeviceType.part_number    |
 | 300    | Highly unique descriptive attribute              | CircuitTermination.xconnect_id, IPAddress.dns_name |
 | 500    | Description                                      | Site.description                                   |
 | 1000   | Custom field default                             | -                                                  |

docs/features/background-jobs.md

@@ -6,7 +6,7 @@ NetBox includes the ability to execute certain functions as background tasks. Th
 * [Custom script](../customization/custom-scripts.md) execution
 * Synchronization of [remote data sources](../integrations/synchronized-data.md)
 
-Additionally, NetBox plugins can enqueue their own background tasks. This is accomplished using the [JobResult model](../models/extras/jobresult.md). Background tasks are executed by the `rqworker` process(es).
+Additionally, NetBox plugins can enqueue their own background tasks. This is accomplished using the [Job model](../models/core/job.md). Background tasks are executed by the `rqworker` process(es).
 
 ## Scheduled Jobs
 

docs/features/circuits.md

@@ -5,13 +5,15 @@ NetBox is ideal for managing your network's transit and peering providers and ci
 ```mermaid
 flowchart TD
     ASN --> Provider
-    Provider --> ProviderNetwork & Circuit
+    Provider --> ProviderNetwork & ProviderAccount & Circuit
+    ProviderAccount --> Circuit
     CircuitType --> Circuit
 
 click ASN "../../models/circuits/asn/"
 click Circuit "../../models/circuits/circuit/"
 click CircuitType "../../models/circuits/circuittype/"
 click Provider "../../models/circuits/provider/"
+click ProviderAccount "../../models/circuits/provideraccount/"
 click ProviderNetwork "../../models/circuits/providernetwork/"
 ```
 
@@ -25,7 +27,7 @@ Sometimes you'll need to model provider networks into which you don't have full
 
 A circuit is a physical connection between two points, which is installed and maintained by an external provider. For example, an Internet connection delivered as a fiber optic cable would be modeled as a circuit in NetBox.
 
-Each circuit is associated with a provider and assigned a circuit ID, which must be unique to that provider. A circuit is also assigned a user-defined type, operational status, and various other operating characteristics.
+Each circuit is associated with a provider and assigned a circuit ID, which must be unique to that provider. A circuit is also assigned a user-defined type, operational status, and various other operating characteristics. Provider accounts can also be employed to further categorize circuits belonging to a common provider: These may represent different business units or technologies.
 
 Each circuit may have up to two terminations (A and Z) defined. Each termination can be associated with a particular site or provider network. In the case of the former, a cable can be connected between the circuit termination and a device component to map its physical connectivity.
 

docs/features/configuration-rendering.md

@@ -12,7 +12,7 @@ click ConfigTemplate "../../models/extras/configtemplate/"
 
 ## Configuration Templates
 
-Configuration templates are written in the [Jinja2 templating language](https://jinja.palletsprojects.com/), and may be automatically populated from remote data sources. Context data is applied to a template during rendering to output a complete configuration file. Below is an example template.
+Configuration templates are written in the [Jinja2 templating language](https://jinja.palletsprojects.com/), and may be automatically populated from remote data sources. Context data is applied to a template during rendering to output a complete configuration file. Below is an example Jinja2 template which renders a simple network switch configuration file.
 
 ```jinja2
 {% extends 'base.j2' %}
@@ -36,3 +36,44 @@ Configuration templates are written in the [Jinja2 templating language](https://
 ```
 
 When rendered for a specific NetBox device, the template's `device` variable will be populated with the device instance, and `ntp_servers` will be pulled from the device's available context data. The resulting output will be a valid configuration segment that can be applied directly to a compatible network device.
+
+## Rendering Templates
+
+### Device Configurations
+
+NetBox provides a REST API endpoint specifically for rendering the default configuration template for a specific device. This is accomplished by sending a POST request to the device's unique URL, optionally including additional context data.
+
+```no-highlight
+curl -X POST \
+-H "Authorization: Token $TOKEN" \
+-H "Content-Type: application/json" \
+-H "Accept: application/json; indent=4" \
+http://netbox:8000/api/dcim/devices/123/render-config/ \
+--data '{
+  "extra_data": "abc123"
+}'
+```
+
+This request will trigger resolution of the device's preferred config template in the following order:
+
+* The config template assigned to the individual device
+* The config template assigned to the device's role
+* The config template assigned to the device's platform
+
+If no config template has been assigned to any of these three objects, the request will fail.
+
+### General Purpose Use
+
+NetBox config templates can also be rendered without being tied to any specific device, using a separate general purpose REST API endpoint. Any data included with a POST request to this endpoint will be passed as context data for the template.
+
+```no-highlight
+curl -X POST \
+-H "Authorization: Token $TOKEN" \
+-H "Content-Type: application/json" \
+-H "Accept: application/json; indent=4" \
+http://netbox:8000/api/extras/config-templates/123/render/ \
+--data '{
+  "foo": "abc",
+  "bar": 123
+}'
+```

docs/features/contacts.md

@@ -31,6 +31,7 @@ The following models support the assignment of contacts:
 
 * circuits.Circuit
 * circuits.Provider
+* circuits.ProviderAccount
 * dcim.Device
 * dcim.Location
 * dcim.Manufacturer

docs/features/synchronized-data.md

@@ -0,0 +1,23 @@
+# Synchronized Data
+
+!!! info "This feature was introduced in NetBox v3.5."
+
+Several models in NetBox support the automatic synchronization of local data from a designated remote source. For example, [configuration templates](./configuration-rendering.md) defined in NetBox can source their content from text files stored in a remote git repository. This accomplished using the core [data source](../models/core/datasource.md) and [data file](../models/core/datafile.md) models.
+
+To enable remote data synchronization, the NetBox administrator first designates one or more remote data sources. NetBox currently supports the following source types:
+
+* Git repository
+* Amazon S3 bucket (or compatible product)
+* Local disk path
+
+(Local disk paths are considered "remote" in this context as they exist outside NetBox's database. These paths could also be mapped to external network shares.)
+
+Each type of remote source has its own configuration parameters. For instance, a git source will ask the user to specify a branch and authentication credentials. Once the source has been created, a synchronization job is run to automatically replicate remote files in the local database.
+
+The following NetBox models can be associated with replicated data files:
+
+* Config contexts
+* Config templates
+* Export templates
+
+Once a data has been designated for a local instance, its data will be replaced with the content of the replicated file. When the replicated file is updated in the future (via synchronization jobs), the local instance will be flagged as having out-of-date data. A user can then synchronize these objects individually or in bulk to effect the update. This two-stgae process ensures that automated synchronization tasks do not immediately affect production data.

docs/getting-started/planning.md

@@ -56,7 +56,7 @@ Below is the (rough) recommended order in which NetBox objects should be created
 4. Manufacturers, device types, and module types
 5. Platforms and device roles
 6. Devices and modules
-7. Providers and provider networks
+7. Providers, provider accounts, and provider networks
 8. Circuit types and circuits
 9. Wireless LAN groups and wireless LANs
 10. Route targets and VRFs

docs/integrations/rest-api.md

@@ -670,5 +670,11 @@ This header specifies the unique ID assigned to the received API request. It can
 GET /api/extras/object-changes/?request_id=e39c84bc-f169-4d5f-bc1c-94487a1b18b5
 ```
 
+The request ID can also be used to filter many objects directly, to return those created or updated by a certain request:
+
+```
+GET /api/dcim/sites/?created_by_request=e39c84bc-f169-4d5f-bc1c-94487a1b18b5
+```
+
 !!! note
     This header is included with _all_ NetBox responses, although it is most practical when working with an API.

docs/models/circuits/circuit.md

@@ -8,6 +8,10 @@ A circuit represents a physical point-to-point data connection, typically used t
 
 The [provider](./provider.md) to which this circuit belongs.
 
+### Provider Account
+
+Circuits may optionally be assigned to a specific [provider account](./provideraccount.md).
+
 ### Circuit ID
 
 An identifier for this circuit. This must be unique to the assigned provider. (Circuits assigned to different providers may have the same circuit ID.)

docs/models/circuits/provider.md

@@ -12,21 +12,10 @@ A unique human-friendly name.
 
 A unique URL-friendly identifier. (This value can be used for filtering.)
 
-### ASN
-
-The AS number assigned to this provider.
-
-!!! warning "Legacy field"
-    This field is being removed in NetBox v3.4. Users are highly encouraged to use the [ASN model](../ipam/asn.md) to track AS number assignment for providers.
-
 ### ASNs
 
 The [AS numbers](../ipam/asn.md) assigned to this provider (optional).
 
-### Account Number
-
-The administrative account identifier tied to this provider for your organization.
-
 ### Portal URL
 
 The URL for the provider's customer service portal.

docs/models/circuits/provideraccount.md

@@ -0,0 +1,17 @@
+# Provider Accounts
+
+This model can be used to represent individual accounts associated with a provider.
+
+## Fields
+
+### Provider
+
+The [provider](./provider.md) the account belongs to.
+
+### Name
+
+A human-friendly name, unique to the provider.
+
+### Account Number
+
+The administrative account identifier tied to this provider for your organization.

docs/models/core/job.md

@@ -1,6 +1,6 @@
-# Job Results
+# Jobs
 
-The JobResult model is used to schedule and record the execution of [background tasks](../../features/background-jobs.md).
+The Job model is used to schedule and record the execution of [background tasks](../../features/background-jobs.md).
 
 ## Fields
 

docs/models/dcim/devicetype.md

@@ -21,6 +21,10 @@ The model number assigned to this device type by its manufacturer. Must be uniqu
 
 A unique URL-friendly representation of the model identifier. (This value can be used for filtering.)
 
+### Default Platform
+
+If defined, devices instantiated from this type will automatically inherit the selected platform. (This assignment can be changed after the device has been created.)
+
 ### Part Number
 
 An alternative part number to uniquely identify the device type.

docs/models/extras/configtemplate.md

@@ -1,6 +1,6 @@
 # Configuration Templates
 
-Configuration templates can be used to render [devices](../dcim/device.md) configurations from [context data](../../features/context-data.md). Templates are written in the [Jinja2 language](https://jinja.palletsprojects.com/) and can be associated with devices roles, platforms, and/or individual devices.
+Configuration templates can be used to render [device](../dcim/device.md) configurations from [context data](../../features/context-data.md). Templates are written in the [Jinja2 language](https://jinja.palletsprojects.com/) and can be associated with devices roles, platforms, and/or individual devices.
 
 Context data is made available to [devices](../dcim/device.md) and/or [virtual machines](../virtualization/virtualmachine.md) based on their relationships to other objects in NetBox. For example, context data can be associated only with devices assigned to a particular site, or only to virtual machines in a certain cluster.
 

docs/plugins/development/forms.md

@@ -70,9 +70,6 @@ class MyModelImportForm(NetBoxModelImportForm):
         fields = ('name', 'status', 'site', 'comments')
 ```
 
-!!! note "Previously NetBoxModelCSVForm"
-    This form class was previously named `NetBoxModelCSVForm`. It was renamed in NetBox v3.4 to convey support for JSON and YAML formats in addition to CSV. The `NetBoxModelCSVForm` class has been retained for backward compatibility and functions exactly the same as `NetBoxModelImportForm`. However, plugin authors should be aware that this backward compatability will be removed in NetBox v3.5.
-
 ### `NetBoxModelBulkEditForm`
 
 This form facilitates editing multiple objects in bulk. Unlike a model form, this form does not have a child `Meta` class, and must explicitly define each field. All fields in a bulk edit form are generally declared with `required=False`.

docs/release-notes/version-3.4.md

@@ -1,5 +1,34 @@
 # NetBox v3.4
 
+## v3.4.8 (FUTURE)
+
+---
+
+## v3.4.7 (2023-03-28)
+
+### Enhancements
+
+* [#11645](https://github.com/netbox-community/netbox/issues/11645) - Automatically set the scheduled time when executing reports/scripts at a recurring interval
+* [#11833](https://github.com/netbox-community/netbox/issues/11833) - Add fieldset support for custom script forms
+* [#11973](https://github.com/netbox-community/netbox/issues/11833) - Use SSID for representing wireless links, if set
+* [#11977](https://github.com/netbox-community/netbox/issues/11977) - Support designating multiple backends via `REMOTE_AUTH_BACKEND` config parameter
+* [#11990](https://github.com/netbox-community/netbox/issues/11990) - Improve error reporting for duplicate CSV column headings
+* [#11991](https://github.com/netbox-community/netbox/issues/11991) - Enable VDC assignment during bulk import/edit of interfaces
+
+### Bug Fixes
+
+* [#11914](https://github.com/netbox-community/netbox/issues/11914) - Include parameters when exporting saved filters
+* [#11933](https://github.com/netbox-community/netbox/issues/11933) - Fix cloning of saved filters
+* [#11984](https://github.com/netbox-community/netbox/issues/11984) - Remove erroneous 802.3az PoE type
+* [#11979](https://github.com/netbox-community/netbox/issues/11979) - Correct URL for tags in route targets list
+* [#12008](https://github.com/netbox-community/netbox/issues/12008) - Enable cloning of export templates
+* [#12029](https://github.com/netbox-community/netbox/issues/12029) - Restore missing description field on virtual chassis form
+* [#12038](https://github.com/netbox-community/netbox/issues/12038) - Correct display of zero values for virtual chassis member priority
+* [#12048](https://github.com/netbox-community/netbox/issues/12048) - Enable cloning of tags
+* [#12058](https://github.com/netbox-community/netbox/issues/12058) - Enable cloning of config contexts
+
+---
+
 ## v3.4.6 (2023-03-13)
 
 ### Enhancements

docs/release-notes/version-3.5.md

@@ -2,11 +2,17 @@
 
 ## v3.5.0 (FUTURE)
 
+### Breaking Changes
+
+* The JobResult model has been moved from the `extras` app to `core` and renamed to Job. Accordingly, its REST API endpoint has been moved from `/api/extras/job-results/` to `/api/core/jobs/`.
+* The `obj_type` field on the Job model (previously JobResult) has been renamed to `object_type` for consistency with other models.
+* The `JOBRESULT_RETENTION` configuration parameter has been renamed to `JOB_RETENTION`.
+
 ### New Features
 
 #### Customizable Dashboard ([#9416](https://github.com/netbox-community/netbox/issues/9416))
 
-The static home view has been replaced with a fully customizable dashboard. Users can construct and rearrange their own personal dashboard to convey the information most pertinent to them. Supported widgets include object statistics, change log records, notes, and more, and we expect to continue adding new widgets over time. Plugins can also register their own custom widgets.
+The static home view has been replaced with a fully customizable dashboard. Users can construct and rearrange their own personal dashboard to convey the information most pertinent to them. Supported widgets include object statistics, configurable object lists, RSS feeds, and notes, and we expect to continue adding new widgets over time.
 
 #### Remote Data Sources ([#11558](https://github.com/netbox-community/netbox/issues/11558))
 
@@ -18,38 +24,86 @@ This release introduces the ability to render device configurations from Jinja2
 
 #### NAPALM Plugin ([#10520](https://github.com/netbox-community/netbox/issues/10520))
 
-The NAPALM integration feature found in previous NetBox releases has been moved from the core application to a dedicated plugin. This allows greater control over the feature's configuration and will unlock additional potential as a separate project.
+The NAPALM integration feature found in previous NetBox releases has been moved from the core application to a [dedicated plugin](https://github.com/netbox-community/netbox-napalm). This allows greater control over the feature's configuration and will unlock additional potential as a separate project.
 
 #### ASN Ranges ([#8550](https://github.com/netbox-community/netbox/issues/8550))
 
 A new ASN range model has been introduced to facilitate the provisioning of new autonomous system numbers from within a prescribed range. For example, an administrator might define an ASN range of 65000-65099 to be used for internal site identification. This includes a REST API endpoint suitable for automatic provisioning, very similar to the allocation of available prefixes and IP addresses.
 
+#### Provider Accounts ([#9047](https://github.com/netbox-community/netbox/issues/9047))
+
+A new model has been introduced to represent individual accounts within a common circuit provider. This replaces the `account` field on the provider model, enabling users to track multiple accounts per provider. New provider account instances will be created automatically during upgrade for all providers which currently have an account assigned. The assignment of individual circuits to a provider account remains optional.
+
+#### Job-Triggered Webhooks ([#8958](https://github.com/netbox-community/netbox/issues/8958))
+
+Two new webhook trigger events have been introduced: `job_start` and `job_end`. These enable users to configure webhook to trigger when a background job starts or ends, respectively. This new functionality can be used, for example, to inform a remote system when a custom script has been executed.
+
 ### Enhancements
 
 * [#7947](https://github.com/netbox-community/netbox/issues/7947) - Enable marking IP ranges as fully utilized
+* [#8184](https://github.com/netbox-community/netbox/issues/8184) - Employ HTMX to dynamically render tables listing related objects
 * [#8272](https://github.com/netbox-community/netbox/issues/8272) - Support bridge relationships among device type interfaces
 * [#8749](https://github.com/netbox-community/netbox/issues/8749) - Support replicating custom field values when cloning an object
-* [#8958](https://github.com/netbox-community/netbox/issues/8958) - Changes in background job status can trigger webhooks
 * [#9073](https://github.com/netbox-community/netbox/issues/9073) - Enable syncing config context data from remote sources
 * [#9653](https://github.com/netbox-community/netbox/issues/9653) - Enable setting a default platform for device types
 * [#10054](https://github.com/netbox-community/netbox/issues/10054) - Introduce advanced object selector for UI forms
-* [#10374](https://github.com/netbox-community/netbox/issues/10374) - Require unique tenant names & slugs per group (not globally)
+* [#10242](https://github.com/netbox-community/netbox/issues/10242) - Redirect to filtered objects list after bulk import
+* [#10374](https://github.com/netbox-community/netbox/issues/10374) - Require unique tenant names & slugs per group
 * [#10729](https://github.com/netbox-community/netbox/issues/10729) - Add date & time custom field type
 * [#11254](https://github.com/netbox-community/netbox/issues/11254) - Introduce the `X-Request-ID` HTTP header to annotate the unique ID of each request for change logging
+* [#11291](https://github.com/netbox-community/netbox/issues/11291) - Optimized GraphQL API request handling
 * [#11440](https://github.com/netbox-community/netbox/issues/11440) - Add an `enabled` field for device type interfaces
+* [#11494](https://github.com/netbox-community/netbox/issues/11494) - Enable filtering objects by create/update request IDs
 * [#11517](https://github.com/netbox-community/netbox/issues/11517) - Standardize the inclusion of related objects across the entire UI
 * [#11584](https://github.com/netbox-community/netbox/issues/11584) - Add a list view for contact assignments
 * [#11625](https://github.com/netbox-community/netbox/issues/11625) - Add HTMX support to ObjectEditView
 * [#11693](https://github.com/netbox-community/netbox/issues/11693) - Enable syncing export template content from remote sources
 * [#11780](https://github.com/netbox-community/netbox/issues/11780) - Enable loading import data from remote sources
+* [#11790](https://github.com/netbox-community/netbox/issues/11790) - Create database indexes for all generic foreign keys
 * [#11968](https://github.com/netbox-community/netbox/issues/11968) - Add navigation menu buttons to create device & VM components
+* [#12068](https://github.com/netbox-community/netbox/issues/12068) - Enable generic foreign key relationships from jobs to NetBox objects
+* [#12085](https://github.com/netbox-community/netbox/issues/12085) - Add a file source view for reports
 
 ### Other Changes
 
 * [#10604](https://github.com/netbox-community/netbox/issues/10604) - Remove unused `extra_tabs` block from `object.html` generic template
 * [#10923](https://github.com/netbox-community/netbox/issues/10923) - Remove unused `NetBoxModelCSVForm` class (replaced by `NetBoxModelImportForm`)
+* [#11489](https://github.com/netbox-community/netbox/issues/11489) - Consoldated several middleware classes
 * [#11611](https://github.com/netbox-community/netbox/issues/11611) - Refactor API viewset classes and introduce NetBoxReadOnlyModelViewSet
 * [#11694](https://github.com/netbox-community/netbox/issues/11694) - Remove obsolete `SmallTextarea` form widget
 * [#11737](https://github.com/netbox-community/netbox/issues/11737) - `ChangeLoggedModel` now inherits `WebhooksMixin`
 * [#11765](https://github.com/netbox-community/netbox/issues/11765) - Retire the `StaticSelect` and `StaticSelectMultiple` form widgets
-* 
+* [#11955](https://github.com/netbox-community/netbox/issues/11955) - Remove the unused `CSVDataField` and `CSVFileField` classes
+* [#12067](https://github.com/netbox-community/netbox/issues/12067) - Move & rename `extras.JobResult` to `core.Job`
+
+### REST API Changes
+
+* All API responses now include a `X-Request-ID` HTTP header indicating the request's unique ID
+* Introduced new endpoints:
+    * `/api/circuits/provider-accounts/`
+    * `/api/core/data-files/`
+    * `/api/core/data-sources/`
+    * `/api/dcim/device/<id>/render-config/`
+    * `/api/extras/config-templates/`
+    * `/api/ipam/asn-ranges/`
+* Removed existing endpoints:
+    * `/api/dcim/device/<id>/napalm/`
+* circuits.Circuit
+    * Added the optional `account` foreign key to ProviderAccount
+* circuits.Provider
+    * Removed the `account` field
+* dcim.DeviceType
+    * Added `default_platform` foreign key (optional)
+* dcim.InterfaceTemplate
+    * Added `enabled` boolean field
+    * Added optional `bridge` foreign key (optional)
+* extras.ConfigContext
+    * Added `data_source`, `data_file`, `data_path`, and `data_synced` fields to enable syncing data from remote sources
+* extras.ExportTemplate
+    * Added `data_source`, `data_file`, `data_path`, and `data_synced` fields to enable syncing content from remote sources
+* extras.Webhook
+    * Added `type_job_start` and `type_job_end` boolean fields
+* ipam.ASN
+    * The `rir` field now fully represents the assigned RIR (if any)
+* ipam.IPRange
+    * Added the `mark_utilized` boolean field (default: false)

mkdocs.yml

@@ -75,6 +75,7 @@ nav:
         - Search: 'features/search.md'
         - Context Data: 'features/context-data.md'
         - Configuration Rendering: 'features/configuration-rendering.md'
+        - Synchronized Data: 'features/synchronized-data.md'
         - Change Logging: 'features/change-logging.md'
         - Journaling: 'features/journaling.md'
         - Background Jobs: 'features/background-jobs.md'
@@ -158,6 +159,7 @@ nav:
         - Core:
             - DataFile: 'models/core/datafile.md'
             - DataSource: 'models/core/datasource.md'
+            - Job: 'models/core/job.md'
         - DCIM:
             - Cable: 'models/dcim/cable.md'
             - ConsolePort: 'models/dcim/consoleport.md'
@@ -207,7 +209,6 @@ nav:
             - CustomLink: 'models/extras/customlink.md'
             - ExportTemplate: 'models/extras/exporttemplate.md'
             - ImageAttachment: 'models/extras/imageattachment.md'
-            - JobResult: 'models/extras/jobresult.md'
             - JournalEntry: 'models/extras/journalentry.md'
             - SavedFilter: 'models/extras/savedfilter.md'
             - StagedChange: 'models/extras/stagedchange.md'

netbox/circuits/api/nested_serializers.py

@@ -11,6 +11,7 @@ __all__ = [
     'NestedCircuitTypeSerializer',
     'NestedProviderNetworkSerializer',
     'NestedProviderSerializer',
+    'NestedProviderAccountSerializer',
 ]
 
 
@@ -42,6 +43,18 @@ class NestedProviderSerializer(WritableNestedSerializer):
         fields = ['id', 'url', 'display', 'name', 'slug', 'circuit_count']
 
 
+#
+# Provider Accounts
+#
+
+class NestedProviderAccountSerializer(WritableNestedSerializer):
+    url = serializers.HyperlinkedIdentityField(view_name='circuits-api:provideraccount-detail')
+
+    class Meta:
+        model = ProviderAccount
+        fields = ['id', 'url', 'display', 'name', 'account']
+
+
 #
 # Circuits
 #

netbox/circuits/api/serializers.py

@@ -18,6 +18,12 @@ from .nested_serializers import *
 
 class ProviderSerializer(NetBoxModelSerializer):
     url = serializers.HyperlinkedIdentityField(view_name='circuits-api:provider-detail')
+    accounts = SerializedPKRelatedField(
+        queryset=ProviderAccount.objects.all(),
+        serializer=NestedProviderAccountSerializer,
+        required=False,
+        many=True
+    )
     asns = SerializedPKRelatedField(
         queryset=ASN.objects.all(),
         serializer=NestedASNSerializer,
@@ -31,11 +37,27 @@ class ProviderSerializer(NetBoxModelSerializer):
     class Meta:
         model = Provider
         fields = [
-            'id', 'url', 'display', 'name', 'slug', 'account', 'description', 'comments', 'asns', 'tags',
+            'id', 'url', 'display', 'name', 'slug', 'accounts', 'description', 'comments', 'asns', 'tags',
             'custom_fields', 'created', 'last_updated', 'circuit_count',
         ]
 
 
+#
+# Provider Accounts
+#
+
+class ProviderAccountSerializer(NetBoxModelSerializer):
+    url = serializers.HyperlinkedIdentityField(view_name='circuits-api:provideraccount-detail')
+    provider = NestedProviderSerializer()
+
+    class Meta:
+        model = ProviderAccount
+        fields = [
+            'id', 'url', 'display', 'provider', 'name', 'account', 'description', 'comments', 'tags', 'custom_fields',
+            'created', 'last_updated',
+        ]
+
+
 #
 # Provider networks
 #
@@ -84,6 +106,7 @@ class CircuitCircuitTerminationSerializer(WritableNestedSerializer):
 class CircuitSerializer(NetBoxModelSerializer):
     url = serializers.HyperlinkedIdentityField(view_name='circuits-api:circuit-detail')
     provider = NestedProviderSerializer()
+    provider_account = NestedProviderAccountSerializer()
     status = ChoiceField(choices=CircuitStatusChoices, required=False)
     type = NestedCircuitTypeSerializer()
     tenant = NestedTenantSerializer(required=False, allow_null=True)
@@ -93,9 +116,9 @@ class CircuitSerializer(NetBoxModelSerializer):
     class Meta:
         model = Circuit
         fields = [
-            'id', 'url', 'display', 'cid', 'provider', 'type', 'status', 'tenant', 'install_date', 'termination_date',
-            'commit_rate', 'description', 'termination_a', 'termination_z', 'comments', 'tags', 'custom_fields',
-            'created', 'last_updated',
+            'id', 'url', 'display', 'cid', 'provider', 'provider_account', 'type', 'status', 'tenant', 'install_date',
+            'termination_date', 'commit_rate', 'description', 'termination_a', 'termination_z', 'comments', 'tags',
+            'custom_fields', 'created', 'last_updated',
         ]
 
 

netbox/circuits/api/urls.py

@@ -7,14 +7,13 @@ router.APIRootView = views.CircuitsRootView
 
 # Providers
 router.register('providers', views.ProviderViewSet)
+router.register('provider-accounts', views.ProviderAccountViewSet)
+router.register('provider-networks', views.ProviderNetworkViewSet)
 
 # Circuits
 router.register('circuit-types', views.CircuitTypeViewSet)
 router.register('circuits', views.CircuitViewSet)
 router.register('circuit-terminations', views.CircuitTerminationViewSet)
 
-# Provider networks
-router.register('provider-networks', views.ProviderNetworkViewSet)
-
 app_name = 'circuits-api'
 urlpatterns = router.urls

netbox/circuits/api/views.py

@@ -46,7 +46,7 @@ class CircuitTypeViewSet(NetBoxModelViewSet):
 
 class CircuitViewSet(NetBoxModelViewSet):
     queryset = Circuit.objects.prefetch_related(
-        'type', 'tenant', 'provider', 'termination_a', 'termination_z'
+        'type', 'tenant', 'provider', 'provider_account', 'termination_a', 'termination_z'
     ).prefetch_related('tags')
     serializer_class = serializers.CircuitSerializer
     filterset_class = filtersets.CircuitFilterSet
@@ -65,6 +65,16 @@ class CircuitTerminationViewSet(PassThroughPortMixin, NetBoxModelViewSet):
     brief_prefetch_fields = ['circuit']
 
 
+#
+# Provider accounts
+#
+
+class ProviderAccountViewSet(NetBoxModelViewSet):
+    queryset = ProviderAccount.objects.prefetch_related('provider', 'tags')
+    serializer_class = serializers.ProviderAccountSerializer
+    filterset_class = filtersets.ProviderAccountFilterSet
+
+
 #
 # Provider networks
 #

netbox/circuits/filtersets.py

@@ -16,6 +16,7 @@ __all__ = (
     'CircuitTerminationFilterSet',
     'CircuitTypeFilterSet',
     'ProviderNetworkFilterSet',
+    'ProviderAccountFilterSet',
     'ProviderFilterSet',
 )
 
@@ -66,7 +67,34 @@ class ProviderFilterSet(NetBoxModelFilterSet, ContactModelFilterSet):
 
     class Meta:
         model = Provider
-        fields = ['id', 'name', 'slug', 'account']
+        fields = ['id', 'name', 'slug']
+
+    def search(self, queryset, name, value):
+        if not value.strip():
+            return queryset
+        return queryset.filter(
+            Q(name__icontains=value) |
+            Q(accounts__account__icontains=value) |
+            Q(accounts__name__icontains=value) |
+            Q(comments__icontains=value)
+        )
+
+
+class ProviderAccountFilterSet(NetBoxModelFilterSet):
+    provider_id = django_filters.ModelMultipleChoiceFilter(
+        queryset=Provider.objects.all(),
+        label=_('Provider (ID)'),
+    )
+    provider = django_filters.ModelMultipleChoiceFilter(
+        field_name='provider__slug',
+        queryset=Provider.objects.all(),
+        to_field_name='slug',
+        label=_('Provider (slug)'),
+    )
+
+    class Meta:
+        model = ProviderAccount
+        fields = ['id', 'name', 'account', 'description']
 
     def search(self, queryset, name, value):
         if not value.strip():
@@ -75,7 +103,7 @@ class ProviderFilterSet(NetBoxModelFilterSet, ContactModelFilterSet):
             Q(name__icontains=value) |
             Q(account__icontains=value) |
             Q(comments__icontains=value)
-        )
+        ).distinct()
 
 
 class ProviderNetworkFilterSet(NetBoxModelFilterSet):
@@ -123,6 +151,11 @@ class CircuitFilterSet(NetBoxModelFilterSet, TenancyFilterSet, ContactModelFilte
         to_field_name='slug',
         label=_('Provider (slug)'),
     )
+    provider_account_id = django_filters.ModelMultipleChoiceFilter(
+        field_name='provider_account',
+        queryset=ProviderAccount.objects.all(),
+        label=_('ProviderAccount (ID)'),
+    )
     provider_network_id = django_filters.ModelMultipleChoiceFilter(
         field_name='terminations__provider_network',
         queryset=ProviderNetwork.objects.all(),

netbox/circuits/forms/bulk_edit.py

@@ -14,6 +14,7 @@ __all__ = (
     'CircuitBulkEditForm',
     'CircuitTypeBulkEditForm',
     'ProviderBulkEditForm',
+    'ProviderAccountBulkEditForm',
     'ProviderNetworkBulkEditForm',
 )
 
@@ -24,11 +25,6 @@ class ProviderBulkEditForm(NetBoxModelBulkEditForm):
         label=_('ASNs'),
         required=False
     )
-    account = forms.CharField(
-        max_length=30,
-        required=False,
-        label=_('Account number')
-    )
     description = forms.CharField(
         max_length=200,
         required=False
@@ -39,10 +35,32 @@ class ProviderBulkEditForm(NetBoxModelBulkEditForm):
 
     model = Provider
     fieldsets = (
-        (None, ('asns', 'account', )),
+        (None, ('asns', 'description')),
     )
     nullable_fields = (
-        'asns', 'account', 'description', 'comments',
+        'asns', 'description', 'comments',
+    )
+
+
+class ProviderAccountBulkEditForm(NetBoxModelBulkEditForm):
+    provider = DynamicModelChoiceField(
+        queryset=Provider.objects.all(),
+        required=False
+    )
+    description = forms.CharField(
+        max_length=200,
+        required=False
+    )
+    comments = CommentField(
+        label=_('Comments')
+    )
+
+    model = ProviderAccount
+    fieldsets = (
+        (None, ('provider', 'description')),
+    )
+    nullable_fields = (
+        'description', 'comments',
     )
 
 
@@ -95,6 +113,13 @@ class CircuitBulkEditForm(NetBoxModelBulkEditForm):
         queryset=Provider.objects.all(),
         required=False
     )
+    provider_account = DynamicModelChoiceField(
+        queryset=ProviderAccount.objects.all(),
+        required=False,
+        query_params={
+            'provider': '$provider'
+        }
+    )
     status = forms.ChoiceField(
         choices=add_blank_choice(CircuitStatusChoices),
         required=False,
@@ -127,7 +152,7 @@ class CircuitBulkEditForm(NetBoxModelBulkEditForm):
     model = Circuit
     fieldsets = (
         ('Circuit', ('provider', 'type', 'status', 'description')),
-        ('Service Parameters', ('install_date', 'termination_date', 'commit_rate')),
+        ('Service Parameters', ('provider_account', 'install_date', 'termination_date', 'commit_rate')),
         ('Tenancy', ('tenant',)),
     )
     nullable_fields = (

netbox/circuits/forms/bulk_import.py

@@ -13,6 +13,7 @@ __all__ = (
     'CircuitTerminationImportForm',
     'CircuitTypeImportForm',
     'ProviderImportForm',
+    'ProviderAccountImportForm',
     'ProviderNetworkImportForm',
 )
 
@@ -23,7 +24,21 @@ class ProviderImportForm(NetBoxModelImportForm):
     class Meta:
         model = Provider
         fields = (
-            'name', 'slug', 'account', 'description', 'comments', 'tags',
+            'name', 'slug', 'description', 'comments', 'tags',
+        )
+
+
+class ProviderAccountImportForm(NetBoxModelImportForm):
+    provider = CSVModelChoiceField(
+        queryset=Provider.objects.all(),
+        to_field_name='name',
+        help_text=_('Assigned provider')
+    )
+
+    class Meta:
+        model = ProviderAccount
+        fields = (
+            'provider', 'name', 'account', 'description', 'comments', 'tags',
         )
 
 
@@ -55,6 +70,11 @@ class CircuitImportForm(NetBoxModelImportForm):
         to_field_name='name',
         help_text=_('Assigned provider')
     )
+    provider_account = CSVModelChoiceField(
+        queryset=ProviderAccount.objects.all(),
+        to_field_name='name',
+        help_text=_('Assigned provider account')
+    )
     type = CSVModelChoiceField(
         queryset=CircuitType.objects.all(),
         to_field_name='name',
@@ -74,8 +94,8 @@ class CircuitImportForm(NetBoxModelImportForm):
     class Meta:
         model = Circuit
         fields = [
-            'cid', 'provider', 'type', 'status', 'tenant', 'install_date', 'termination_date', 'commit_rate',
-            'description', 'comments', 'tags'
+            'cid', 'provider', 'provider_account', 'type', 'status', 'tenant', 'install_date', 'termination_date',
+            'commit_rate', 'description', 'comments', 'tags'
         ]
 
 

netbox/circuits/forms/filtersets.py

@@ -13,6 +13,7 @@ __all__ = (
     'CircuitFilterForm',
     'CircuitTypeFilterForm',
     'ProviderFilterForm',
+    'ProviderAccountFilterForm',
     'ProviderNetworkFilterForm',
 )
 
@@ -56,6 +57,23 @@ class ProviderFilterForm(ContactModelFilterForm, NetBoxModelFilterSetForm):
     tag = TagFilterField(model)
 
 
+class ProviderAccountFilterForm(NetBoxModelFilterSetForm):
+    model = ProviderAccount
+    fieldsets = (
+        (None, ('q', 'filter_id', 'tag')),
+        ('Attributes', ('provider_id', 'account')),
+    )
+    provider_id = DynamicModelMultipleChoiceField(
+        queryset=Provider.objects.all(),
+        required=False,
+        label=_('Provider')
+    )
+    account = forms.CharField(
+        required=False
+    )
+    tag = TagFilterField(model)
+
+
 class ProviderNetworkFilterForm(NetBoxModelFilterSetForm):
     model = ProviderNetwork
     fieldsets = (
@@ -83,7 +101,7 @@ class CircuitFilterForm(TenancyFilterForm, ContactModelFilterForm, NetBoxModelFi
     model = Circuit
     fieldsets = (
         (None, ('q', 'filter_id', 'tag')),
-        ('Provider', ('provider_id', 'provider_network_id')),
+        ('Provider', ('provider_id', 'provider_account_id', 'provider_network_id')),
         ('Attributes', ('type_id', 'status', 'install_date', 'termination_date', 'commit_rate')),
         ('Location', ('region_id', 'site_group_id', 'site_id')),
         ('Tenant', ('tenant_group_id', 'tenant_id')),
@@ -99,6 +117,14 @@ class CircuitFilterForm(TenancyFilterForm, ContactModelFilterForm, NetBoxModelFi
         required=False,
         label=_('Provider')
     )
+    provider_account_id = DynamicModelMultipleChoiceField(
+        queryset=ProviderAccount.objects.all(),
+        required=False,
+        query_params={
+            'provider_id': '$provider_id'
+        },
+        label=_('Provider account')
+    )
     provider_network_id = DynamicModelMultipleChoiceField(
         queryset=ProviderNetwork.objects.all(),
         required=False,

netbox/circuits/forms/model_forms.py

@@ -14,6 +14,7 @@ __all__ = (
     'CircuitTerminationForm',
     'CircuitTypeForm',
     'ProviderForm',
+    'ProviderAccountForm',
     'ProviderNetworkForm',
 )
 
@@ -29,13 +30,25 @@ class ProviderForm(NetBoxModelForm):
 
     fieldsets = (
         ('Provider', ('name', 'slug', 'asns', 'description', 'tags')),
-        ('Support Info', ('account',)),
     )
 
     class Meta:
         model = Provider
         fields = [
-            'name', 'slug', 'account', 'asns', 'description', 'comments', 'tags',
+            'name', 'slug', 'asns', 'description', 'comments', 'tags',
+        ]
+
+
+class ProviderAccountForm(NetBoxModelForm):
+    provider = DynamicModelChoiceField(
+        queryset=Provider.objects.all()
+    )
+    comments = CommentField()
+
+    class Meta:
+        model = ProviderAccount
+        fields = [
+            'provider', 'name', 'account', 'description', 'comments', 'tags',
         ]
 
 
@@ -74,7 +87,15 @@ class CircuitTypeForm(NetBoxModelForm):
 
 class CircuitForm(TenancyForm, NetBoxModelForm):
     provider = DynamicModelChoiceField(
-        queryset=Provider.objects.all()
+        queryset=Provider.objects.all(),
+        selector=True
+    )
+    provider_account = DynamicModelChoiceField(
+        queryset=ProviderAccount.objects.all(),
+        required=False,
+        query_params={
+            'provider_id': '$provider',
+        }
     )
     type = DynamicModelChoiceField(
         queryset=CircuitType.objects.all()
@@ -82,7 +103,7 @@ class CircuitForm(TenancyForm, NetBoxModelForm):
     comments = CommentField()
 
     fieldsets = (
-        ('Circuit', ('provider', 'cid', 'type', 'status', 'description', 'tags')),
+        ('Circuit', ('provider', 'provider_account', 'cid', 'type', 'status', 'description', 'tags')),
         ('Service Parameters', ('install_date', 'termination_date', 'commit_rate')),
         ('Tenancy', ('tenant_group', 'tenant')),
     )
@@ -90,8 +111,8 @@ class CircuitForm(TenancyForm, NetBoxModelForm):
     class Meta:
         model = Circuit
         fields = [
-            'cid', 'type', 'provider', 'status', 'install_date', 'termination_date', 'commit_rate', 'description',
-            'tenant_group', 'tenant', 'comments', 'tags',
+            'cid', 'type', 'provider', 'provider_account', 'status', 'install_date', 'termination_date', 'commit_rate',
+            'description', 'tenant_group', 'tenant', 'comments', 'tags',
         ]
         widgets = {
             'install_date': DatePicker(),
@@ -101,18 +122,9 @@ class CircuitForm(TenancyForm, NetBoxModelForm):
 
 
 class CircuitTerminationForm(NetBoxModelForm):
-    provider = DynamicModelChoiceField(
-        queryset=Provider.objects.all(),
-        required=False,
-        initial_params={
-            'circuits': '$circuit'
-        }
-    )
     circuit = DynamicModelChoiceField(
         queryset=Circuit.objects.all(),
-        query_params={
-            'provider_id': '$provider',
-        },
+        selector=True
     )
     site = DynamicModelChoiceField(
         queryset=Site.objects.all(),
@@ -128,8 +140,8 @@ class CircuitTerminationForm(NetBoxModelForm):
     class Meta:
         model = CircuitTermination
         fields = [
-            'provider', 'circuit', 'term_side', 'site', 'provider_network', 'mark_connected', 'port_speed',
-            'upstream_speed', 'xconnect_id', 'pp_info', 'description', 'tags',
+            'circuit', 'term_side', 'site', 'provider_network', 'mark_connected', 'port_speed', 'upstream_speed',
+            'xconnect_id', 'pp_info', 'description', 'tags',
         ]
         widgets = {
             'port_speed': SelectSpeedWidget(),

netbox/circuits/graphql/schema.py

@@ -1,21 +1,41 @@
 import graphene
 
+from circuits import models
 from netbox.graphql.fields import ObjectField, ObjectListField
 from .types import *
+from utilities.graphql_optimizer import gql_query_optimizer
 
 
 class CircuitsQuery(graphene.ObjectType):
     circuit = ObjectField(CircuitType)
     circuit_list = ObjectListField(CircuitType)
 
+    def resolve_circuit_list(root, info, **kwargs):
+        return gql_query_optimizer(models.Circuit.objects.all(), info)
+
     circuit_termination = ObjectField(CircuitTerminationType)
     circuit_termination_list = ObjectListField(CircuitTerminationType)
 
+    def resolve_circuit_termination_list(root, info, **kwargs):
+        return gql_query_optimizer(models.CircuitTermination.objects.all(), info)
+
     circuit_type = ObjectField(CircuitTypeType)
     circuit_type_list = ObjectListField(CircuitTypeType)
 
+    def resolve_circuit_type_list(root, info, **kwargs):
+        return gql_query_optimizer(models.CircuitType.objects.all(), info)
+
     provider = ObjectField(ProviderType)
     provider_list = ObjectListField(ProviderType)
 
+    def resolve_provider_list(root, info, **kwargs):
+        return gql_query_optimizer(models.Provider.objects.all(), info)
+
+    provider_account = ObjectField(ProviderAccountType)
+    provider_account_list = ObjectListField(ProviderAccountType)
+
     provider_network = ObjectField(ProviderNetworkType)
     provider_network_list = ObjectListField(ProviderNetworkType)
+
+    def resolve_provider_network_list(root, info, **kwargs):
+        return gql_query_optimizer(models.ProviderNetwork.objects.all(), info)

netbox/circuits/graphql/types.py

@@ -10,6 +10,7 @@ __all__ = (
     'CircuitType',
     'CircuitTypeType',
     'ProviderType',
+    'ProviderAccountType',
     'ProviderNetworkType',
 )
 
@@ -45,6 +46,14 @@ class ProviderType(NetBoxObjectType, ContactsMixin):
         filterset_class = filtersets.ProviderFilterSet
 
 
+class ProviderAccountType(NetBoxObjectType):
+
+    class Meta:
+        model = models.ProviderAccount
+        fields = '__all__'
+        filterset_class = filtersets.ProviderAccountFilterSet
+
+
 class ProviderNetworkType(NetBoxObjectType):
 
     class Meta:

netbox/circuits/migrations/0042_provideraccount.py

@@ -0,0 +1,91 @@
+from django.db import migrations, models
+import django.db.models.deletion
+import taggit.managers
+import utilities.json
+
+
+def create_provideraccounts_from_providers(apps, schema_editor):
+    """
+    Migrate Account in Provider model to separate account model
+    """
+    Provider = apps.get_model('circuits', 'Provider')
+    ProviderAccount = apps.get_model('circuits', 'ProviderAccount')
+
+    provider_accounts = []
+    for provider in Provider.objects.all():
+        if provider.account:
+            provider_accounts.append(ProviderAccount(
+                provider=provider,
+                account=provider.account
+            ))
+    ProviderAccount.objects.bulk_create(provider_accounts, batch_size=100)
+
+
+def restore_providers_from_provideraccounts(apps, schema_editor):
+    """
+    Restore Provider account values from auto-generated ProviderAccounts
+    """
+    ProviderAccount = apps.get_model('circuits', 'ProviderAccount')
+    provider_accounts = ProviderAccount.objects.order_by('pk')
+    for provideraccount in provider_accounts:
+        if provider_accounts.filter(provider=provideraccount.provider)[0] == provideraccount:
+            provideraccount.provider.account = provideraccount.account
+            provideraccount.provider.save()
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('extras', '0084_staging'),
+        ('circuits', '0041_standardize_description_comments'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='ProviderAccount',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False)),
+                ('created', models.DateTimeField(auto_now_add=True, null=True)),
+                ('last_updated', models.DateTimeField(auto_now=True, null=True)),
+                ('custom_field_data', models.JSONField(blank=True, default=dict, encoder=utilities.json.CustomFieldJSONEncoder)),
+                ('description', models.CharField(blank=True, max_length=200)),
+                ('comments', models.TextField(blank=True)),
+                ('account', models.CharField(max_length=100)),
+                ('name', models.CharField(blank=True, max_length=100)),
+                ('provider', models.ForeignKey(on_delete=django.db.models.deletion.PROTECT, related_name='accounts', to='circuits.provider')),
+                ('tags', taggit.managers.TaggableManager(through='extras.TaggedItem', to='extras.Tag')),
+            ],
+            options={
+                'ordering': ('provider', 'account'),
+            },
+        ),
+        migrations.AddConstraint(
+            model_name='provideraccount',
+            constraint=models.UniqueConstraint(condition=models.Q(('name', ''), _negated=True), fields=('provider', 'name'), name='circuits_provideraccount_unique_provider_name'),
+        ),
+        migrations.AddConstraint(
+            model_name='provideraccount',
+            constraint=models.UniqueConstraint(fields=('provider', 'account'), name='circuits_provideraccount_unique_provider_account'),
+        ),
+        migrations.RunPython(
+            create_provideraccounts_from_providers, restore_providers_from_provideraccounts
+        ),
+        migrations.RemoveField(
+            model_name='provider',
+            name='account',
+        ),
+        migrations.AddField(
+            model_name='circuit',
+            name='provider_account',
+            field=models.ForeignKey(on_delete=django.db.models.deletion.PROTECT, related_name='circuits', to='circuits.provideraccount', null=True, blank=True),
+            preserve_default=False,
+        ),
+        migrations.AlterModelOptions(
+            name='circuit',
+            options={'ordering': ['provider', 'provider_account', 'cid']},
+        ),
+        migrations.AddConstraint(
+            model_name='circuit',
+            constraint=models.UniqueConstraint(fields=('provider_account', 'cid'), name='circuits_circuit_unique_provideraccount_cid'),
+        ),
+    ]

netbox/circuits/models/circuits.py

@@ -29,8 +29,8 @@ class CircuitType(OrganizationalModel):
 class Circuit(PrimaryModel):
     """
     A communications circuit connects two points. Each Circuit belongs to a Provider; Providers may have multiple
-    circuits. Each circuit is also assigned a CircuitType and a Site.  Circuit port speed and commit rate are measured
-    in Kbps.
+    circuits. Each circuit is also assigned a CircuitType and a Site, and may optionally be assigned to a particular
+    ProviderAccount. Circuit port speed and commit rate are measured in Kbps.
     """
     cid = models.CharField(
         max_length=100,
@@ -42,6 +42,13 @@ class Circuit(PrimaryModel):
         on_delete=models.PROTECT,
         related_name='circuits'
     )
+    provider_account = models.ForeignKey(
+        to='circuits.ProviderAccount',
+        on_delete=models.PROTECT,
+        related_name='circuits',
+        blank=True,
+        null=True
+    )
     type = models.ForeignKey(
         to='CircuitType',
         on_delete=models.PROTECT,
@@ -103,7 +110,8 @@ class Circuit(PrimaryModel):
     )
 
     clone_fields = (
-        'provider', 'type', 'status', 'tenant', 'install_date', 'termination_date', 'commit_rate', 'description',
+        'provider', 'provider_account', 'type', 'status', 'tenant', 'install_date', 'termination_date', 'commit_rate',
+        'description',
     )
     prerequisite_models = (
         'circuits.CircuitType',
@@ -111,12 +119,16 @@ class Circuit(PrimaryModel):
     )
 
     class Meta:
-        ordering = ['provider', 'cid']
+        ordering = ['provider', 'provider_account', 'cid']
         constraints = (
             models.UniqueConstraint(
                 fields=('provider', 'cid'),
                 name='%(app_label)s_%(class)s_unique_provider_cid'
             ),
+            models.UniqueConstraint(
+                fields=('provider_account', 'cid'),
+                name='%(app_label)s_%(class)s_unique_provideraccount_cid'
+            ),
         )
 
     def __str__(self):
@@ -128,6 +140,12 @@ class Circuit(PrimaryModel):
     def get_status_color(self):
         return CircuitStatusChoices.colors.get(self.status)
 
+    def clean(self):
+        super().clean()
+
+        if self.provider_account and self.provider != self.provider_account.provider:
+            raise ValidationError({'provider_account': "The assigned account must belong to the assigned provider."})
+
 
 class CircuitTermination(
     CustomFieldsMixin,

netbox/circuits/models/providers.py

@@ -1,5 +1,6 @@
 from django.contrib.contenttypes.fields import GenericRelation
 from django.db import models
+from django.db.models import Q
 from django.urls import reverse
 from django.utils.translation import gettext as _
 
@@ -8,6 +9,7 @@ from netbox.models import PrimaryModel
 __all__ = (
     'ProviderNetwork',
     'Provider',
+    'ProviderAccount',
 )
 
 
@@ -30,20 +32,13 @@ class Provider(PrimaryModel):
         related_name='providers',
         blank=True
     )
-    account = models.CharField(
-        max_length=30,
-        blank=True,
-        verbose_name='Account number'
-    )
 
     # Generic relations
     contacts = GenericRelation(
         to='tenancy.ContactAssignment'
     )
 
-    clone_fields = (
-        'account',
-    )
+    clone_fields = ()
 
     class Meta:
         ordering = ['name']
@@ -55,6 +50,54 @@ class Provider(PrimaryModel):
         return reverse('circuits:provider', args=[self.pk])
 
 
+class ProviderAccount(PrimaryModel):
+    """
+    This is a discrete account within a provider.  Each Circuit belongs to a Provider Account.
+    """
+    provider = models.ForeignKey(
+        to='circuits.Provider',
+        on_delete=models.PROTECT,
+        related_name='accounts'
+    )
+    account = models.CharField(
+        max_length=100,
+        verbose_name='Account ID'
+    )
+    name = models.CharField(
+        max_length=100,
+        blank=True
+    )
+
+    # Generic relations
+    contacts = GenericRelation(
+        to='tenancy.ContactAssignment'
+    )
+
+    clone_fields = ('provider', )
+
+    class Meta:
+        ordering = ('provider', 'account')
+        constraints = (
+            models.UniqueConstraint(
+                fields=('provider', 'account'),
+                name='%(app_label)s_%(class)s_unique_provider_account'
+            ),
+            models.UniqueConstraint(
+                fields=('provider', 'name'),
+                name='%(app_label)s_%(class)s_unique_provider_name',
+                condition=~Q(name="")
+            ),
+        )
+
+    def __str__(self):
+        if self.name:
+            return f'{self.account} ({self.name})'
+        return f'{self.account}'
+
+    def get_absolute_url(self):
+        return reverse('circuits:provideraccount', args=[self.pk])
+
+
 class ProviderNetwork(PrimaryModel):
     """
     This represents a provider network which exists outside of NetBox, the details of which are unknown or

netbox/circuits/search.py

@@ -39,12 +39,20 @@ class ProviderIndex(SearchIndex):
     model = models.Provider
     fields = (
         ('name', 100),
-        ('account', 200),
         ('description', 500),
         ('comments', 5000),
     )
 
 
+class ProviderAccountIndex(SearchIndex):
+    model = models.ProviderAccount
+    fields = (
+        ('name', 100),
+        ('account', 200),
+        ('comments', 5000),
+    )
+
+
 @register_search
 class ProviderNetworkIndex(SearchIndex):
     model = models.ProviderNetwork

netbox/circuits/tables/circuits.py

@@ -1,4 +1,5 @@
 import django_tables2 as tables
+
 from circuits.models import *
 from tenancy.tables import ContactsColumnMixin, TenancyColumnsMixin
 
@@ -50,6 +51,10 @@ class CircuitTable(TenancyColumnsMixin, ContactsColumnMixin, NetBoxTable):
     provider = tables.Column(
         linkify=True
     )
+    provider_account = tables.Column(
+        linkify=True,
+        verbose_name='Account'
+    )
     status = columns.ChoiceFieldColumn()
     termination_a = tables.TemplateColumn(
         template_code=CIRCUITTERMINATION_LINK,
@@ -68,9 +73,9 @@ class CircuitTable(TenancyColumnsMixin, ContactsColumnMixin, NetBoxTable):
     class Meta(NetBoxTable.Meta):
         model = Circuit
         fields = (
-            'pk', 'id', 'cid', 'provider', 'type', 'status', 'tenant', 'tenant_group', 'termination_a', 'termination_z',
-            'install_date', 'termination_date', 'commit_rate', 'description', 'comments', 'contacts', 'tags', 'created',
-            'last_updated',
+            'pk', 'id', 'cid', 'provider', 'provider_account', 'type', 'status', 'tenant', 'tenant_group',
+            'termination_a', 'termination_z', 'install_date', 'termination_date', 'commit_rate', 'description',
+            'comments', 'contacts', 'tags', 'created', 'last_updated',
         )
         default_columns = (
             'pk', 'cid', 'provider', 'type', 'status', 'tenant', 'termination_a', 'termination_z', 'description',

netbox/circuits/tables/providers.py

@@ -7,6 +7,7 @@ from netbox.tables import NetBoxTable, columns
 
 __all__ = (
     'ProviderTable',
+    'ProviderAccountTable',
     'ProviderNetworkTable',
 )
 
@@ -15,6 +16,16 @@ class ProviderTable(ContactsColumnMixin, NetBoxTable):
     name = tables.Column(
         linkify=True
     )
+    accounts = columns.ManyToManyColumn(
+        linkify_item=True,
+        verbose_name='Accounts'
+    )
+    account_count = columns.LinkedCountColumn(
+        accessor=tables.A('accounts__count'),
+        viewname='circuits:provideraccount_list',
+        url_params={'account_id': 'pk'},
+        verbose_name='Account Count'
+    )
     asns = columns.ManyToManyColumn(
         linkify_item=True,
         verbose_name='ASNs'
@@ -39,10 +50,38 @@ class ProviderTable(ContactsColumnMixin, NetBoxTable):
     class Meta(NetBoxTable.Meta):
         model = Provider
         fields = (
-            'pk', 'id', 'name', 'asns', 'account', 'asn_count', 'circuit_count', 'description', 'comments', 'contacts',
-            'tags', 'created', 'last_updated',
+            'pk', 'id', 'name', 'accounts', 'account_count', 'asns', 'asn_count', 'circuit_count', 'description',
+            'comments', 'contacts', 'tags', 'created', 'last_updated',
         )
-        default_columns = ('pk', 'name', 'account', 'circuit_count')
+        default_columns = ('pk', 'name', 'account_count', 'circuit_count')
+
+
+class ProviderAccountTable(ContactsColumnMixin, NetBoxTable):
+    account = tables.Column(
+        linkify=True
+    )
+    name = tables.Column()
+    provider = tables.Column(
+        linkify=True
+    )
+    circuit_count = columns.LinkedCountColumn(
+        accessor=Accessor('count_circuits'),
+        viewname='circuits:circuit_list',
+        url_params={'provider_account_id': 'pk'},
+        verbose_name='Circuits'
+    )
+    comments = columns.MarkdownColumn()
+    tags = columns.TagColumn(
+        url_name='circuits:provideraccount_list'
+    )
+
+    class Meta(NetBoxTable.Meta):
+        model = ProviderAccount
+        fields = (
+            'pk', 'id', 'account', 'name', 'provider', 'circuit_count', 'comments', 'contacts', 'tags', 'created',
+            'last_updated',
+        )
+        default_columns = ('pk', 'account', 'name', 'provider', 'circuit_count')
 
 
 class ProviderNetworkTable(NetBoxTable):

netbox/circuits/tests/test_api.py

@@ -20,7 +20,7 @@ class ProviderTest(APIViewTestCases.APIViewTestCase):
     model = Provider
     brief_fields = ['circuit_count', 'display', 'id', 'name', 'slug', 'url']
     bulk_update_data = {
-        'account': '1234',
+        'comments': 'New comments',
     }
 
     @classmethod
@@ -106,6 +106,12 @@ class CircuitTest(APIViewTestCases.APIViewTestCase):
         )
         Provider.objects.bulk_create(providers)
 
+        provider_accounts = (
+            ProviderAccount(name='Provider Account 1', provider=providers[0], account='1234'),
+            ProviderAccount(name='Provider Account 2', provider=providers[1], account='2345'),
+        )
+        ProviderAccount.objects.bulk_create(provider_accounts)
+
         circuit_types = (
             CircuitType(name='Circuit Type 1', slug='circuit-type-1'),
             CircuitType(name='Circuit Type 2', slug='circuit-type-2'),
@@ -113,9 +119,9 @@ class CircuitTest(APIViewTestCases.APIViewTestCase):
         CircuitType.objects.bulk_create(circuit_types)
 
         circuits = (
-            Circuit(cid='Circuit 1', provider=providers[0], type=circuit_types[0]),
-            Circuit(cid='Circuit 2', provider=providers[0], type=circuit_types[0]),
-            Circuit(cid='Circuit 3', provider=providers[0], type=circuit_types[0]),
+            Circuit(cid='Circuit 1', provider=providers[0], provider_account=provider_accounts[0], type=circuit_types[0]),
+            Circuit(cid='Circuit 2', provider=providers[0], provider_account=provider_accounts[0], type=circuit_types[0]),
+            Circuit(cid='Circuit 3', provider=providers[0], provider_account=provider_accounts[0], type=circuit_types[0]),
         )
         Circuit.objects.bulk_create(circuits)
 
@@ -123,16 +129,19 @@ class CircuitTest(APIViewTestCases.APIViewTestCase):
             {
                 'cid': 'Circuit 4',
                 'provider': providers[1].pk,
+                'provider_account': provider_accounts[1].pk,
                 'type': circuit_types[1].pk,
             },
             {
                 'cid': 'Circuit 5',
                 'provider': providers[1].pk,
+                'provider_account': provider_accounts[1].pk,
                 'type': circuit_types[1].pk,
             },
             {
                 'cid': 'Circuit 6',
                 'provider': providers[1].pk,
+                'provider_account': provider_accounts[1].pk,
                 'type': circuit_types[1].pk,
             },
         ]
@@ -197,6 +206,49 @@ class CircuitTerminationTest(APIViewTestCases.APIViewTestCase):
         }
 
 
+class ProviderAccountTest(APIViewTestCases.APIViewTestCase):
+    model = ProviderAccount
+    brief_fields = ['account', 'display', 'id', 'name', 'url']
+
+    @classmethod
+    def setUpTestData(cls):
+        providers = (
+            Provider(name='Provider 1', slug='provider-1'),
+            Provider(name='Provider 2', slug='provider-2'),
+        )
+        Provider.objects.bulk_create(providers)
+
+        provider_accounts = (
+            ProviderAccount(name='Provider Account 1', provider=providers[0], account='1234'),
+            ProviderAccount(name='Provider Account 2', provider=providers[0], account='2345'),
+            ProviderAccount(name='Provider Account 3', provider=providers[0], account='3456'),
+        )
+        ProviderAccount.objects.bulk_create(provider_accounts)
+
+        cls.create_data = [
+            {
+                'name': 'Provider Account 4',
+                'provider': providers[0].pk,
+                'account': '4567',
+            },
+            {
+                'name': 'Provider Account 5',
+                'provider': providers[0].pk,
+                'account': '5678',
+            },
+            {
+                'name': 'Provider Account 6',
+                'provider': providers[0].pk,
+                'account': '6789',
+            },
+        ]
+
+        cls.bulk_update_data = {
+            'provider': providers[1].pk,
+            'description': 'New description',
+        }
+
+
 class ProviderNetworkTest(APIViewTestCases.APIViewTestCase):
     model = ProviderNetwork
     brief_fields = ['display', 'id', 'name', 'url']

netbox/circuits/tests/test_filtersets.py

@@ -25,11 +25,11 @@ class ProviderTestCase(TestCase, ChangeLoggedFilterSetTests):
         ASN.objects.bulk_create(asns)
 
         providers = (
-            Provider(name='Provider 1', slug='provider-1', account='1234'),
-            Provider(name='Provider 2', slug='provider-2', account='2345'),
-            Provider(name='Provider 3', slug='provider-3', account='3456'),
-            Provider(name='Provider 4', slug='provider-4', account='4567'),
-            Provider(name='Provider 5', slug='provider-5', account='5678'),
+            Provider(name='Provider 1', slug='provider-1'),
+            Provider(name='Provider 2', slug='provider-2'),
+            Provider(name='Provider 3', slug='provider-3'),
+            Provider(name='Provider 4', slug='provider-4'),
+            Provider(name='Provider 5', slug='provider-5'),
         )
         Provider.objects.bulk_create(providers)
         providers[0].asns.set([asns[0]])
@@ -64,8 +64,8 @@ class ProviderTestCase(TestCase, ChangeLoggedFilterSetTests):
         CircuitType.objects.bulk_create(circuit_types)
 
         circuits = (
-            Circuit(provider=providers[0], type=circuit_types[0], cid='Test Circuit 1'),
-            Circuit(provider=providers[1], type=circuit_types[1], cid='Test Circuit 1'),
+            Circuit(provider=providers[0], type=circuit_types[0], cid='Circuit 1'),
+            Circuit(provider=providers[1], type=circuit_types[1], cid='Circuit 2'),
         )
         Circuit.objects.bulk_create(circuits)
 
@@ -87,10 +87,6 @@ class ProviderTestCase(TestCase, ChangeLoggedFilterSetTests):
         params = {'asn_id': [asns[0].pk, asns[1].pk]}
         self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
 
-    def test_account(self):
-        params = {'account': ['1234', '2345']}
-        self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
-
     def test_region(self):
         regions = Region.objects.all()[:2]
         params = {'region_id': [regions[0].pk, regions[1].pk]}
@@ -193,9 +189,17 @@ class CircuitTestCase(TestCase, ChangeLoggedFilterSetTests):
         providers = (
             Provider(name='Provider 1', slug='provider-1'),
             Provider(name='Provider 2', slug='provider-2'),
+            Provider(name='Provider 3', slug='provider-3'),
         )
         Provider.objects.bulk_create(providers)
 
+        provider_accounts = (
+            ProviderAccount(name='Provider Account 1', provider=providers[0], account='A'),
+            ProviderAccount(name='Provider Account 2', provider=providers[1], account='B'),
+            ProviderAccount(name='Provider Account 3', provider=providers[2], account='C'),
+        )
+        ProviderAccount.objects.bulk_create(provider_accounts)
+
         provider_networks = (
             ProviderNetwork(name='Provider Network 1', provider=providers[1]),
             ProviderNetwork(name='Provider Network 2', provider=providers[1]),
@@ -204,12 +208,12 @@ class CircuitTestCase(TestCase, ChangeLoggedFilterSetTests):
         ProviderNetwork.objects.bulk_create(provider_networks)
 
         circuits = (
-            Circuit(provider=providers[0], tenant=tenants[0], type=circuit_types[0], cid='Test Circuit 1', install_date='2020-01-01', termination_date='2021-01-01', commit_rate=1000, status=CircuitStatusChoices.STATUS_ACTIVE, description='foobar1'),
-            Circuit(provider=providers[0], tenant=tenants[0], type=circuit_types[0], cid='Test Circuit 2', install_date='2020-01-02', termination_date='2021-01-02', commit_rate=2000, status=CircuitStatusChoices.STATUS_ACTIVE, description='foobar2'),
-            Circuit(provider=providers[0], tenant=tenants[1], type=circuit_types[0], cid='Test Circuit 3', install_date='2020-01-03', termination_date='2021-01-03', commit_rate=3000, status=CircuitStatusChoices.STATUS_PLANNED),
-            Circuit(provider=providers[1], tenant=tenants[1], type=circuit_types[1], cid='Test Circuit 4', install_date='2020-01-04', termination_date='2021-01-04', commit_rate=4000, status=CircuitStatusChoices.STATUS_PLANNED),
-            Circuit(provider=providers[1], tenant=tenants[2], type=circuit_types[1], cid='Test Circuit 5', install_date='2020-01-05', termination_date='2021-01-05', commit_rate=5000, status=CircuitStatusChoices.STATUS_OFFLINE),
-            Circuit(provider=providers[1], tenant=tenants[2], type=circuit_types[1], cid='Test Circuit 6', install_date='2020-01-06', termination_date='2021-01-06', commit_rate=6000, status=CircuitStatusChoices.STATUS_OFFLINE),
+            Circuit(provider=providers[0], provider_account=provider_accounts[0], tenant=tenants[0], type=circuit_types[0], cid='Test Circuit 1', install_date='2020-01-01', termination_date='2021-01-01', commit_rate=1000, status=CircuitStatusChoices.STATUS_ACTIVE, description='foobar1'),
+            Circuit(provider=providers[0], provider_account=provider_accounts[0], tenant=tenants[0], type=circuit_types[0], cid='Test Circuit 2', install_date='2020-01-02', termination_date='2021-01-02', commit_rate=2000, status=CircuitStatusChoices.STATUS_ACTIVE, description='foobar2'),
+            Circuit(provider=providers[0], provider_account=provider_accounts[1], tenant=tenants[1], type=circuit_types[0], cid='Test Circuit 3', install_date='2020-01-03', termination_date='2021-01-03', commit_rate=3000, status=CircuitStatusChoices.STATUS_PLANNED),
+            Circuit(provider=providers[1], provider_account=provider_accounts[1], tenant=tenants[1], type=circuit_types[1], cid='Test Circuit 4', install_date='2020-01-04', termination_date='2021-01-04', commit_rate=4000, status=CircuitStatusChoices.STATUS_PLANNED),
+            Circuit(provider=providers[1], provider_account=provider_accounts[2], tenant=tenants[2], type=circuit_types[1], cid='Test Circuit 5', install_date='2020-01-05', termination_date='2021-01-05', commit_rate=5000, status=CircuitStatusChoices.STATUS_OFFLINE),
+            Circuit(provider=providers[1], provider_account=provider_accounts[2], tenant=tenants[2], type=circuit_types[1], cid='Test Circuit 6', install_date='2020-01-06', termination_date='2021-01-06', commit_rate=6000, status=CircuitStatusChoices.STATUS_OFFLINE),
         )
         Circuit.objects.bulk_create(circuits)
 
@@ -246,6 +250,11 @@ class CircuitTestCase(TestCase, ChangeLoggedFilterSetTests):
         params = {'provider': [provider.slug]}
         self.assertEqual(self.filterset(params, self.queryset).qs.count(), 3)
 
+    def test_provider_account(self):
+        provider_accounts = ProviderAccount.objects.all()[:2]
+        params = {'provider_account_id': [provider_accounts[0].pk, provider_accounts[1].pk]}
+        self.assertEqual(self.filterset(params, self.queryset).qs.count(), 4)
+
     def test_provider_network(self):
         provider_networks = ProviderNetwork.objects.all()[:2]
         params = {'provider_network_id': [provider_networks[0].pk, provider_networks[1].pk]}
@@ -445,3 +454,44 @@ class ProviderNetworkTestCase(TestCase, ChangeLoggedFilterSetTests):
         self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
         params = {'provider': [providers[0].slug, providers[1].slug]}
         self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
+
+
+class ProviderAccountTestCase(TestCase, ChangeLoggedFilterSetTests):
+    queryset = ProviderAccount.objects.all()
+    filterset = ProviderAccountFilterSet
+
+    @classmethod
+    def setUpTestData(cls):
+
+        providers = (
+            Provider(name='Provider 1', slug='provider-1'),
+            Provider(name='Provider 2', slug='provider-2'),
+            Provider(name='Provider 3', slug='provider-3'),
+        )
+        Provider.objects.bulk_create(providers)
+
+        provider_accounts = (
+            ProviderAccount(name='Provider Account 1', provider=providers[0], description='foobar1', account='1234'),
+            ProviderAccount(name='Provider Account 2', provider=providers[1], description='foobar2', account='2345'),
+            ProviderAccount(name='Provider Account 3', provider=providers[2], account='3456'),
+        )
+        ProviderAccount.objects.bulk_create(provider_accounts)
+
+    def test_name(self):
+        params = {'name': ['Provider Account 1', 'Provider Account 2']}
+        self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
+
+    def test_account(self):
+        params = {'account': ['1234', '3456']}
+        self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
+
+    def test_description(self):
+        params = {'description': ['foobar1', 'foobar2']}
+        self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
+
+    def test_provider(self):
+        providers = Provider.objects.all()[:2]
+        params = {'provider_id': [providers[0].pk, providers[1].pk]}
+        self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
+        params = {'provider': [providers[0].slug, providers[1].slug]}
+        self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)

netbox/circuits/tests/test_views.py

@@ -38,7 +38,6 @@ class ProviderTestCase(ViewTestCases.PrimaryObjectViewTestCase):
             'name': 'Provider X',
             'slug': 'provider-x',
             'asns': [asns[6].pk, asns[7].pk],
-            'account': '1234',
             'comments': 'Another provider',
             'tags': [t.pk for t in tags],
         }
@@ -58,7 +57,6 @@ class ProviderTestCase(ViewTestCases.PrimaryObjectViewTestCase):
         )
 
         cls.bulk_edit_data = {
-            'account': '5678',
             'comments': 'New comments',
         }
 
@@ -124,6 +122,12 @@ class CircuitTestCase(ViewTestCases.PrimaryObjectViewTestCase):
         )
         Provider.objects.bulk_create(providers)
 
+        provider_accounts = (
+            ProviderAccount(name='Provider Account 1', provider=providers[0], account='1234'),
+            ProviderAccount(name='Provider Account 2', provider=providers[1], account='2345'),
+        )
+        ProviderAccount.objects.bulk_create(provider_accounts)
+
         circuittypes = (
             CircuitType(name='Circuit Type 1', slug='circuit-type-1'),
             CircuitType(name='Circuit Type 2', slug='circuit-type-2'),
@@ -131,9 +135,9 @@ class CircuitTestCase(ViewTestCases.PrimaryObjectViewTestCase):
         CircuitType.objects.bulk_create(circuittypes)
 
         circuits = (
-            Circuit(cid='Circuit 1', provider=providers[0], type=circuittypes[0]),
-            Circuit(cid='Circuit 2', provider=providers[0], type=circuittypes[0]),
-            Circuit(cid='Circuit 3', provider=providers[0], type=circuittypes[0]),
+            Circuit(cid='Circuit 1', provider=providers[0], provider_account=provider_accounts[0], type=circuittypes[0]),
+            Circuit(cid='Circuit 2', provider=providers[0], provider_account=provider_accounts[0], type=circuittypes[0]),
+            Circuit(cid='Circuit 3', provider=providers[0], provider_account=provider_accounts[0], type=circuittypes[0]),
         )
 
         Circuit.objects.bulk_create(circuits)
@@ -143,6 +147,7 @@ class CircuitTestCase(ViewTestCases.PrimaryObjectViewTestCase):
         cls.form_data = {
             'cid': 'Circuit X',
             'provider': providers[1].pk,
+            'provider_account': provider_accounts[1].pk,
             'type': circuittypes[1].pk,
             'status': CircuitStatusChoices.STATUS_DECOMMISSIONED,
             'tenant': None,
@@ -155,10 +160,10 @@ class CircuitTestCase(ViewTestCases.PrimaryObjectViewTestCase):
         }
 
         cls.csv_data = (
-            "cid,provider,type,status",
-            "Circuit 4,Provider 1,Circuit Type 1,active",
-            "Circuit 5,Provider 1,Circuit Type 1,active",
-            "Circuit 6,Provider 1,Circuit Type 1,active",
+            "cid,provider,provider_account,type,status",
+            "Circuit 4,Provider 1,Provider Account 1,Circuit Type 1,active",
+            "Circuit 5,Provider 1,Provider Account 1,Circuit Type 1,active",
+            "Circuit 6,Provider 1,Provider Account 1,Circuit Type 1,active",
         )
 
         cls.csv_update_data = (
@@ -170,6 +175,7 @@ class CircuitTestCase(ViewTestCases.PrimaryObjectViewTestCase):
 
         cls.bulk_edit_data = {
             'provider': providers[1].pk,
+            'provider_account': provider_accounts[1].pk,
             'type': circuittypes[1].pk,
             'status': CircuitStatusChoices.STATUS_DECOMMISSIONED,
             'tenant': None,
@@ -179,6 +185,57 @@ class CircuitTestCase(ViewTestCases.PrimaryObjectViewTestCase):
         }
 
 
+class ProviderAccountTestCase(ViewTestCases.PrimaryObjectViewTestCase):
+    model = ProviderAccount
+
+    @classmethod
+    def setUpTestData(cls):
+
+        providers = (
+            Provider(name='Provider 1', slug='provider-1'),
+            Provider(name='Provider 2', slug='provider-2'),
+        )
+        Provider.objects.bulk_create(providers)
+
+        provider_accounts = (
+            ProviderAccount(name='Provider Account 1', provider=providers[0], account='1234'),
+            ProviderAccount(name='Provider Account 2', provider=providers[0], account='2345'),
+            ProviderAccount(name='Provider Account 3', provider=providers[0], account='3456'),
+        )
+        ProviderAccount.objects.bulk_create(provider_accounts)
+
+        tags = create_tags('Alpha', 'Bravo', 'Charlie')
+
+        cls.form_data = {
+            'name': 'Provider Account X',
+            'provider': providers[1].pk,
+            'account': 'XXXX',
+            'description': 'A new provider network',
+            'comments': 'Longer description goes here',
+            'tags': [t.pk for t in tags],
+        }
+
+        cls.csv_data = (
+            "name,provider,account,description",
+            "Provider Account 4,Provider 1,4567,Foo",
+            "Provider Account 5,Provider 1,5678,Bar",
+            "Provider Account 6,Provider 1,6789,Baz",
+        )
+
+        cls.csv_update_data = (
+            "id,name,account,description",
+            f"{provider_accounts[0].pk},Provider Network 7,7890,New description7",
+            f"{provider_accounts[1].pk},Provider Network 8,8901,New description8",
+            f"{provider_accounts[2].pk},Provider Network 9,9012,New description9",
+        )
+
+        cls.bulk_edit_data = {
+            'provider': providers[1].pk,
+            'description': 'New description',
+            'comments': 'New comments',
+        }
+
+
 class ProviderNetworkTestCase(ViewTestCases.PrimaryObjectViewTestCase):
     model = ProviderNetwork
 

netbox/circuits/urls.py

@@ -14,6 +14,14 @@ urlpatterns = [
     path('providers/delete/', views.ProviderBulkDeleteView.as_view(), name='provider_bulk_delete'),
     path('providers/<int:pk>/', include(get_model_urls('circuits', 'provider'))),
 
+    # Provider accounts
+    path('provider-accounts/', views.ProviderAccountListView.as_view(), name='provideraccount_list'),
+    path('provider-accounts/add/', views.ProviderAccountEditView.as_view(), name='provideraccount_add'),
+    path('provider-accounts/import/', views.ProviderAccountBulkImportView.as_view(), name='provideraccount_import'),
+    path('provider-accounts/edit/', views.ProviderAccountBulkEditView.as_view(), name='provideraccount_bulk_edit'),
+    path('provider-accounts/delete/', views.ProviderAccountBulkDeleteView.as_view(), name='provideraccount_bulk_delete'),
+    path('provider-accounts/<int:pk>/', include(get_model_urls('circuits', 'provideraccount'))),
+
     # Provider networks
     path('provider-networks/', views.ProviderNetworkListView.as_view(), name='providernetwork_list'),
     path('provider-networks/add/', views.ProviderNetworkEditView.as_view(), name='providernetwork_add'),

netbox/circuits/views.py

@@ -31,6 +31,7 @@ class ProviderView(generic.ObjectView):
 
     def get_extra_context(self, request, instance):
         related_models = (
+            (ProviderAccount.objects.restrict(request.user, 'view').filter(provider=instance), 'provider_id'),
             (Circuit.objects.restrict(request.user, 'view').filter(provider=instance), 'provider_id'),
         )
 
@@ -53,7 +54,6 @@ class ProviderDeleteView(generic.ObjectDeleteView):
 class ProviderBulkImportView(generic.BulkImportView):
     queryset = Provider.objects.all()
     model_form = forms.ProviderImportForm
-    table = tables.ProviderTable
 
 
 class ProviderBulkEditView(generic.BulkEditView):
@@ -73,6 +73,67 @@ class ProviderBulkDeleteView(generic.BulkDeleteView):
     table = tables.ProviderTable
 
 
+#
+# ProviderAccounts
+#
+
+class ProviderAccountListView(generic.ObjectListView):
+    queryset = ProviderAccount.objects.annotate(
+        count_circuits=count_related(Circuit, 'provider_account')
+    )
+    filterset = filtersets.ProviderAccountFilterSet
+    filterset_form = forms.ProviderAccountFilterForm
+    table = tables.ProviderAccountTable
+
+
+@register_model_view(ProviderAccount)
+class ProviderAccountView(generic.ObjectView):
+    queryset = ProviderAccount.objects.all()
+
+    def get_extra_context(self, request, instance):
+        related_models = (
+            (Circuit.objects.restrict(request.user, 'view').filter(provider_account=instance), 'provider_account_id'),
+        )
+
+        return {
+            'related_models': related_models,
+        }
+
+
+@register_model_view(ProviderAccount, 'edit')
+class ProviderAccountEditView(generic.ObjectEditView):
+    queryset = ProviderAccount.objects.all()
+    form = forms.ProviderAccountForm
+
+
+@register_model_view(ProviderAccount, 'delete')
+class ProviderAccountDeleteView(generic.ObjectDeleteView):
+    queryset = ProviderAccount.objects.all()
+
+
+class ProviderAccountBulkImportView(generic.BulkImportView):
+    queryset = ProviderAccount.objects.all()
+    model_form = forms.ProviderAccountImportForm
+    table = tables.ProviderAccountTable
+
+
+class ProviderAccountBulkEditView(generic.BulkEditView):
+    queryset = ProviderAccount.objects.annotate(
+        count_circuits=count_related(Circuit, 'provider_account')
+    )
+    filterset = filtersets.ProviderAccountFilterSet
+    table = tables.ProviderAccountTable
+    form = forms.ProviderAccountBulkEditForm
+
+
+class ProviderAccountBulkDeleteView(generic.BulkDeleteView):
+    queryset = ProviderAccount.objects.annotate(
+        count_circuits=count_related(Circuit, 'provider_account')
+    )
+    filterset = filtersets.ProviderAccountFilterSet
+    table = tables.ProviderAccountTable
+
+
 #
 # Provider networks
 #
@@ -115,7 +176,6 @@ class ProviderNetworkDeleteView(generic.ObjectDeleteView):
 class ProviderNetworkBulkImportView(generic.BulkImportView):
     queryset = ProviderNetwork.objects.all()
     model_form = forms.ProviderNetworkImportForm
-    table = tables.ProviderNetworkTable
 
 
 class ProviderNetworkBulkEditView(generic.BulkEditView):
@@ -172,7 +232,6 @@ class CircuitTypeDeleteView(generic.ObjectDeleteView):
 class CircuitTypeBulkImportView(generic.BulkImportView):
     queryset = CircuitType.objects.all()
     model_form = forms.CircuitTypeImportForm
-    table = tables.CircuitTypeTable
 
 
 class CircuitTypeBulkEditView(generic.BulkEditView):
@@ -224,7 +283,6 @@ class CircuitDeleteView(generic.ObjectDeleteView):
 class CircuitBulkImportView(generic.BulkImportView):
     queryset = Circuit.objects.all()
     model_form = forms.CircuitImportForm
-    table = tables.CircuitTable
     additional_permissions = [
         'circuits.add_circuittermination',
     ]

netbox/core/api/nested_serializers.py

@@ -1,12 +1,16 @@
 from rest_framework import serializers
 
+from core.choices import JobStatusChoices
 from core.models import *
+from netbox.api.fields import ChoiceField
 from netbox.api.serializers import WritableNestedSerializer
+from users.api.nested_serializers import NestedUserSerializer
 
-__all__ = [
+__all__ = (
     'NestedDataFileSerializer',
     'NestedDataSourceSerializer',
-]
+    'NestedJobSerializer',
+)
 
 
 class NestedDataSourceSerializer(WritableNestedSerializer):
@@ -23,3 +27,15 @@ class NestedDataFileSerializer(WritableNestedSerializer):
     class Meta:
         model = DataFile
         fields = ['id', 'url', 'display', 'path']
+
+
+class NestedJobSerializer(serializers.ModelSerializer):
+    url = serializers.HyperlinkedIdentityField(view_name='core-api:job-detail')
+    status = ChoiceField(choices=JobStatusChoices)
+    user = NestedUserSerializer(
+        read_only=True
+    )
+
+    class Meta:
+        model = Job
+        fields = ['url', 'created', 'completed', 'user', 'status']

netbox/core/api/serializers.py

@@ -2,12 +2,15 @@ from rest_framework import serializers
 
 from core.choices import *
 from core.models import *
-from netbox.api.fields import ChoiceField
-from netbox.api.serializers import NetBoxModelSerializer
+from netbox.api.fields import ChoiceField, ContentTypeField
+from netbox.api.serializers import BaseModelSerializer, NetBoxModelSerializer
+from users.api.nested_serializers import NestedUserSerializer
 from .nested_serializers import *
 
 __all__ = (
+    'DataFileSerializer',
     'DataSourceSerializer',
+    'JobSerializer',
 )
 
 
@@ -49,3 +52,21 @@ class DataFileSerializer(NetBoxModelSerializer):
         fields = [
             'id', 'url', 'display', 'source', 'path', 'last_updated', 'size', 'hash',
         ]
+
+
+class JobSerializer(BaseModelSerializer):
+    url = serializers.HyperlinkedIdentityField(view_name='core-api:job-detail')
+    user = NestedUserSerializer(
+        read_only=True
+    )
+    status = ChoiceField(choices=JobStatusChoices, read_only=True)
+    object_type = ContentTypeField(
+        read_only=True
+    )
+
+    class Meta:
+        model = Job
+        fields = [
+            'id', 'url', 'display', 'object_type', 'object_id', 'name', 'status', 'created', 'scheduled', 'interval',
+            'started', 'completed', 'user', 'data', 'job_id',
+        ]

netbox/core/api/urls.py

@@ -9,5 +9,8 @@ router.APIRootView = views.CoreRootView
 router.register('data-sources', views.DataSourceViewSet)
 router.register('data-files', views.DataFileViewSet)
 
+# Jobs
+router.register('jobs', views.JobViewSet)
+
 app_name = 'core-api'
 urlpatterns = router.urls

netbox/core/api/views.py

@@ -4,6 +4,7 @@ from rest_framework.decorators import action
 from rest_framework.exceptions import PermissionDenied
 from rest_framework.response import Response
 from rest_framework.routers import APIRootView
+from rest_framework.viewsets import ReadOnlyModelViewSet
 
 from core import filtersets
 from core.models import *
@@ -20,10 +21,6 @@ class CoreRootView(APIRootView):
         return 'Core'
 
 
-#
-# Data sources
-#
-
 class DataSourceViewSet(NetBoxModelViewSet):
     queryset = DataSource.objects.annotate(
         file_count=count_related(DataFile, 'source')
@@ -50,3 +47,12 @@ class DataFileViewSet(NetBoxReadOnlyModelViewSet):
     queryset = DataFile.objects.defer('data').prefetch_related('source')
     serializer_class = serializers.DataFileSerializer
     filterset_class = filtersets.DataFileFilterSet
+
+
+class JobViewSet(ReadOnlyModelViewSet):
+    """
+    Retrieve a list of job results
+    """
+    queryset = Job.objects.prefetch_related('user')
+    serializer_class = serializers.JobSerializer
+    filterset_class = filtersets.JobFilterSet

netbox/core/choices.py

@@ -20,7 +20,6 @@ class DataSourceTypeChoices(ChoiceSet):
 
 
 class DataSourceStatusChoices(ChoiceSet):
-
     NEW = 'new'
     QUEUED = 'queued'
     SYNCING = 'syncing'
@@ -34,3 +33,46 @@ class DataSourceStatusChoices(ChoiceSet):
         (COMPLETED, _('Completed'), 'green'),
         (FAILED, _('Failed'), 'red'),
     )
+
+
+#
+# Managed files
+#
+
+class ManagedFileRootPathChoices(ChoiceSet):
+    SCRIPTS = 'scripts'  # settings.SCRIPTS_ROOT
+    REPORTS = 'reports'  # settings.REPORTS_ROOT
+
+    CHOICES = (
+        (SCRIPTS, _('Scripts')),
+        (REPORTS, _('Reports')),
+    )
+
+
+#
+# Jobs
+#
+
+class JobStatusChoices(ChoiceSet):
+
+    STATUS_PENDING = 'pending'
+    STATUS_SCHEDULED = 'scheduled'
+    STATUS_RUNNING = 'running'
+    STATUS_COMPLETED = 'completed'
+    STATUS_ERRORED = 'errored'
+    STATUS_FAILED = 'failed'
+
+    CHOICES = (
+        (STATUS_PENDING, 'Pending', 'cyan'),
+        (STATUS_SCHEDULED, 'Scheduled', 'gray'),
+        (STATUS_RUNNING, 'Running', 'blue'),
+        (STATUS_COMPLETED, 'Completed', 'green'),
+        (STATUS_ERRORED, 'Errored', 'red'),
+        (STATUS_FAILED, 'Failed', 'red'),
+    )
+
+    TERMINAL_STATE_CHOICES = (
+        STATUS_COMPLETED,
+        STATUS_ERRORED,
+        STATUS_FAILED,
+    )

netbox/core/data_backends.py

@@ -91,15 +91,14 @@ class GitBackend(DataBackend):
         username = self.params.get('username')
         password = self.params.get('password')
         if username and password:
-            url_components = list(urlparse(self.url))
-            # Prepend username & password to netloc
-            url_components[1] = quote(f'{username}@{password}:') + url_components[1]
-            url = urlunparse(url_components)
+            # Add username & password to URL
+            parsed = urlparse(self.url)
+            url = f'{parsed.scheme}://{quote(username)}:{quote(password)}@{parsed.netloc}{parsed.path}'
         else:
             url = self.url
 
         # Compile git arguments
-        args = ['git', 'clone', '--depth', '1']
+        args = [settings.GIT_PATH, 'clone', '--depth', '1']
         if branch := self.params.get('branch'):
             args.extend(['--branch', branch])
         args.extend([url, local_path.name])
@@ -112,10 +111,13 @@ class GitBackend(DataBackend):
         logger.debug(f"Cloning git repo: {' '.join(args)}")
         try:
             subprocess.run(args, check=True, capture_output=True, env=env_vars)
-        except subprocess.CalledProcessError as e:
+        except FileNotFoundError as e:
             raise SyncError(
-                f"Fetching remote data failed: {e.stderr}"
+                f"Unable to fetch: git executable not found. Check that the git executable exists at the "
+                f"configured path: {settings.GIT_PATH}"
             )
+        except subprocess.CalledProcessError as e:
+            raise SyncError(f"Fetching remote data failed: {e.stderr}")
 
         yield local_path.name
 

netbox/core/filtersets.py

@@ -3,13 +3,14 @@ from django.utils.translation import gettext as _
 
 import django_filters
 
-from netbox.filtersets import ChangeLoggedModelFilterSet, NetBoxModelFilterSet
+from netbox.filtersets import BaseFilterSet, ChangeLoggedModelFilterSet, NetBoxModelFilterSet
 from .choices import *
 from .models import *
 
 __all__ = (
     'DataFileFilterSet',
     'DataSourceFilterSet',
+    'JobFilterSet',
 )
 
 
@@ -62,3 +63,62 @@ class DataFileFilterSet(ChangeLoggedModelFilterSet):
         return queryset.filter(
             Q(path__icontains=value)
         )
+
+
+class JobFilterSet(BaseFilterSet):
+    q = django_filters.CharFilter(
+        method='search',
+        label=_('Search'),
+    )
+    created = django_filters.DateTimeFilter()
+    created__before = django_filters.DateTimeFilter(
+        field_name='created',
+        lookup_expr='lte'
+    )
+    created__after = django_filters.DateTimeFilter(
+        field_name='created',
+        lookup_expr='gte'
+    )
+    scheduled = django_filters.DateTimeFilter()
+    scheduled__before = django_filters.DateTimeFilter(
+        field_name='scheduled',
+        lookup_expr='lte'
+    )
+    scheduled__after = django_filters.DateTimeFilter(
+        field_name='scheduled',
+        lookup_expr='gte'
+    )
+    started = django_filters.DateTimeFilter()
+    started__before = django_filters.DateTimeFilter(
+        field_name='started',
+        lookup_expr='lte'
+    )
+    started__after = django_filters.DateTimeFilter(
+        field_name='started',
+        lookup_expr='gte'
+    )
+    completed = django_filters.DateTimeFilter()
+    completed__before = django_filters.DateTimeFilter(
+        field_name='completed',
+        lookup_expr='lte'
+    )
+    completed__after = django_filters.DateTimeFilter(
+        field_name='completed',
+        lookup_expr='gte'
+    )
+    status = django_filters.MultipleChoiceFilter(
+        choices=JobStatusChoices,
+        null_value=None
+    )
+
+    class Meta:
+        model = Job
+        fields = ('id', 'object_type', 'object_id', 'name', 'interval', 'status', 'user')
+
+    def search(self, queryset, name, value):
+        if not value.strip():
+            return queryset
+        return queryset.filter(
+            Q(user__username__icontains=value) |
+            Q(name__icontains=value)
+        )

netbox/core/forms/bulk_edit.py

@@ -27,7 +27,6 @@ class DataSourceBulkEditForm(NetBoxModelBulkEditForm):
         required=False
     )
     comments = CommentField(
-        widget=forms.Textarea,
         label=_('Comments')
     )
     parameters = forms.JSONField(

netbox/core/forms/filtersets.py

@@ -1,14 +1,22 @@
 from django import forms
+from django.contrib.auth.models import User
+from django.contrib.contenttypes.models import ContentType
 from django.utils.translation import gettext as _
 
 from core.choices import *
 from core.models import *
+from extras.forms.mixins import SavedFiltersMixin
+from extras.utils import FeatureQuery
 from netbox.forms import NetBoxModelFilterSetForm
-from utilities.forms import BOOLEAN_WITH_BLANK_CHOICES, DynamicModelMultipleChoiceField
+from utilities.forms import (
+    APISelectMultiple, BOOLEAN_WITH_BLANK_CHOICES, ContentTypeChoiceField, DateTimePicker,
+    DynamicModelMultipleChoiceField, FilterForm,
+)
 
 __all__ = (
     'DataFileFilterForm',
     'DataSourceFilterForm',
+    'JobFilterForm',
 )
 
 
@@ -45,3 +53,63 @@ class DataFileFilterForm(NetBoxModelFilterSetForm):
         required=False,
         label=_('Data source')
     )
+
+
+class JobFilterForm(SavedFiltersMixin, FilterForm):
+    fieldsets = (
+        (None, ('q', 'filter_id')),
+        ('Attributes', ('object_type', 'status')),
+        ('Creation', (
+            'created__before', 'created__after', 'scheduled__before', 'scheduled__after', 'started__before',
+            'started__after', 'completed__before', 'completed__after', 'user',
+        )),
+    )
+    object_type = ContentTypeChoiceField(
+        label=_('Object Type'),
+        queryset=ContentType.objects.filter(FeatureQuery('jobs').get_query()),
+        required=False,
+    )
+    status = forms.MultipleChoiceField(
+        choices=JobStatusChoices,
+        required=False
+    )
+    created__after = forms.DateTimeField(
+        required=False,
+        widget=DateTimePicker()
+    )
+    created__before = forms.DateTimeField(
+        required=False,
+        widget=DateTimePicker()
+    )
+    scheduled__after = forms.DateTimeField(
+        required=False,
+        widget=DateTimePicker()
+    )
+    scheduled__before = forms.DateTimeField(
+        required=False,
+        widget=DateTimePicker()
+    )
+    started__after = forms.DateTimeField(
+        required=False,
+        widget=DateTimePicker()
+    )
+    started__before = forms.DateTimeField(
+        required=False,
+        widget=DateTimePicker()
+    )
+    completed__after = forms.DateTimeField(
+        required=False,
+        widget=DateTimePicker()
+    )
+    completed__before = forms.DateTimeField(
+        required=False,
+        widget=DateTimePicker()
+    )
+    user = DynamicModelMultipleChoiceField(
+        queryset=User.objects.all(),
+        required=False,
+        label=_('User'),
+        widget=APISelectMultiple(
+            api_url='/api/users/users/',
+        )
+    )

netbox/core/forms/model_forms.py

@@ -3,12 +3,14 @@ import copy
 from django import forms
 
 from core.models import *
+from extras.forms.mixins import SyncedDataMixin
 from netbox.forms import NetBoxModelForm
 from netbox.registry import registry
 from utilities.forms import CommentField, get_field_value
 
 __all__ = (
     'DataSourceForm',
+    'ManagedFileForm',
 )
 
 
@@ -73,3 +75,37 @@ class DataSourceForm(NetBoxModelForm):
         self.instance.parameters = parameters
 
         return super().save(*args, **kwargs)
+
+
+class ManagedFileForm(SyncedDataMixin, NetBoxModelForm):
+    upload_file = forms.FileField(
+        required=False
+    )
+
+    fieldsets = (
+        ('File Upload', ('upload_file',)),
+        ('Data Source', ('data_source', 'data_file')),
+    )
+
+    class Meta:
+        model = ManagedFile
+        fields = ('data_source', 'data_file')
+
+    def clean(self):
+        super().clean()
+
+        if self.cleaned_data.get('upload_file') and self.cleaned_data.get('data_file'):
+            raise forms.ValidationError("Cannot upload a file and sync from an existing file")
+        if not self.cleaned_data.get('upload_file') and not self.cleaned_data.get('data_file'):
+            raise forms.ValidationError("Must upload a file or select a data file to sync")
+
+        return self.cleaned_data
+
+    def save(self, *args, **kwargs):
+        # If a file was uploaded, save it to disk
+        if self.cleaned_data['upload_file']:
+            self.instance.file_path = self.cleaned_data['upload_file'].name
+            with open(self.instance.full_path, 'wb+') as new_file:
+                new_file.write(self.cleaned_data['upload_file'].read())
+
+        return super().save(*args, **kwargs)

netbox/core/graphql/schema.py

@@ -1,12 +1,20 @@
 import graphene
 
+from core import models
 from netbox.graphql.fields import ObjectField, ObjectListField
 from .types import *
+from utilities.graphql_optimizer import gql_query_optimizer
 
 
 class CoreQuery(graphene.ObjectType):
     data_file = ObjectField(DataFileType)
     data_file_list = ObjectListField(DataFileType)
 
+    def resolve_data_file_list(root, info, **kwargs):
+        return gql_query_optimizer(models.DataFile.objects.all(), info)
+
     data_source = ObjectField(DataSourceType)
     data_source_list = ObjectListField(DataSourceType)
+
+    def resolve_data_source_list(root, info, **kwargs):
+        return gql_query_optimizer(models.DataSource.objects.all(), info)

netbox/core/jobs.py

@@ -1,30 +1,30 @@
 import logging
 
-from extras.choices import JobResultStatusChoices
 from netbox.search.backends import search_backend
 from .choices import *
 from .exceptions import SyncError
 from .models import DataSource
+from rq.timeouts import JobTimeoutException
 
 logger = logging.getLogger(__name__)
 
 
-def sync_datasource(job_result, *args, **kwargs):
+def sync_datasource(job, *args, **kwargs):
     """
     Call sync() on a DataSource.
     """
-    datasource = DataSource.objects.get(name=job_result.name)
+    datasource = DataSource.objects.get(pk=job.object_id)
 
     try:
-        job_result.start()
+        job.start()
         datasource.sync()
 
         # Update the search cache for DataFiles belonging to this source
         search_backend.cache(datasource.datafiles.iterator())
 
-        job_result.terminate()
+        job.terminate()
 
-    except SyncError as e:
-        job_result.terminate(status=JobResultStatusChoices.STATUS_ERRORED)
+    except (SyncError, JobTimeoutException) as e:
+        job.terminate(status=JobStatusChoices.STATUS_ERRORED)
         DataSource.objects.filter(pk=datasource.pk).update(status=DataSourceStatusChoices.FAILED)
         logging.error(e)

netbox/core/management/commands/makemigrations.py

@@ -4,7 +4,7 @@ from django.core.management.base import CommandError
 from django.core.management.commands.makemigrations import Command as _Command
 from django.db import models
 
-from . import custom_deconstruct
+from utilities.migration import custom_deconstruct
 
 models.Field.deconstruct = custom_deconstruct
 

netbox/core/management/commands/migrate.py

@@ -2,6 +2,6 @@
 from django.core.management.commands.migrate import Command
 from django.db import models
 
-from . import custom_deconstruct
+from utilities.migration import custom_deconstruct
 
 models.Field.deconstruct = custom_deconstruct

netbox/core/migrations/0002_managedfile.py

@@ -0,0 +1,39 @@
+# Generated by Django 4.1.7 on 2023-03-23 17:35
+
+from django.db import migrations, models
+import django.db.models.deletion
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('core', '0001_initial'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='ManagedFile',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False)),
+                ('data_path', models.CharField(blank=True, editable=False, max_length=1000)),
+                ('data_synced', models.DateTimeField(blank=True, editable=False, null=True)),
+                ('created', models.DateTimeField(auto_now_add=True)),
+                ('last_updated', models.DateTimeField(blank=True, editable=False, null=True)),
+                ('file_root', models.CharField(max_length=1000)),
+                ('file_path', models.FilePathField(editable=False)),
+                ('data_file', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='+', to='core.datafile')),
+                ('data_source', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.PROTECT, related_name='+', to='core.datasource')),
+            ],
+            options={
+                'ordering': ('file_root', 'file_path'),
+            },
+        ),
+        migrations.AddIndex(
+            model_name='managedfile',
+            index=models.Index(fields=['file_root', 'file_path'], name='core_managedfile_root_path'),
+        ),
+        migrations.AddConstraint(
+            model_name='managedfile',
+            constraint=models.UniqueConstraint(fields=('file_root', 'file_path'), name='core_managedfile_unique_root_path'),
+        ),
+    ]

netbox/core/migrations/0003_job.py

@@ -0,0 +1,40 @@
+# Generated by Django 4.1.7 on 2023-03-27 15:02
+
+from django.conf import settings
+import django.core.validators
+from django.db import migrations, models
+import django.db.models.deletion
+import extras.utils
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('contenttypes', '0002_remove_content_type_name'),
+        migrations.swappable_dependency(settings.AUTH_USER_MODEL),
+        ('core', '0002_managedfile'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='Job',
+            fields=[
+                ('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False)),
+                ('object_id', models.PositiveBigIntegerField(blank=True, null=True)),
+                ('name', models.CharField(max_length=200)),
+                ('created', models.DateTimeField()),
+                ('scheduled', models.DateTimeField(blank=True, null=True)),
+                ('interval', models.PositiveIntegerField(blank=True, null=True, validators=[django.core.validators.MinValueValidator(1)])),
+                ('started', models.DateTimeField(blank=True, null=True)),
+                ('completed', models.DateTimeField(blank=True, null=True)),
+                ('status', models.CharField(default='pending', max_length=30)),
+                ('data', models.JSONField(blank=True, null=True)),
+                ('job_id', models.UUIDField(unique=True)),
+                ('object_type', models.ForeignKey(limit_choices_to=extras.utils.FeatureQuery('jobs'), on_delete=django.db.models.deletion.CASCADE, related_name='jobs', to='contenttypes.contenttype')),
+                ('user', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='+', to=settings.AUTH_USER_MODEL)),
+            ],
+            options={
+                'ordering': ['-created'],
+            },
+        ),
+    ]

netbox/core/migrations/0004_replicate_jobresults.py

@@ -0,0 +1,45 @@
+from django.db import migrations
+
+
+def replicate_jobresults(apps, schema_editor):
+    """
+    Replicate existing JobResults to the new Jobs table before deleting the old JobResults table.
+    """
+    Job = apps.get_model('core', 'Job')
+    JobResult = apps.get_model('extras', 'JobResult')
+
+    jobs = []
+    for job_result in JobResult.objects.order_by('pk').iterator(chunk_size=100):
+        jobs.append(
+            Job(
+                object_type=job_result.obj_type,
+                name=job_result.name,
+                created=job_result.created,
+                scheduled=job_result.scheduled,
+                interval=job_result.interval,
+                started=job_result.started,
+                completed=job_result.completed,
+                user=job_result.user,
+                status=job_result.status,
+                data=job_result.data,
+                job_id=job_result.job_id,
+            )
+        )
+        if len(jobs) == 100:
+            Job.objects.bulk_create(jobs)
+    if jobs:
+        Job.objects.bulk_create(jobs)
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('core', '0003_job'),
+    ]
+
+    operations = [
+        migrations.RunPython(
+            code=replicate_jobresults,
+            reverse_code=migrations.RunPython.noop
+        ),
+    ]

netbox/core/migrations/0005_job_created_auto_now.py

@@ -0,0 +1,18 @@
+# Generated by Django 4.1.7 on 2023-03-27 17:28
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('core', '0004_replicate_jobresults'),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name='job',
+            name='created',
+            field=models.DateTimeField(auto_now_add=True),
+        ),
+    ]

netbox/core/models/__init__.py

@@ -1 +1,3 @@
 from .data import *
+from .files import *
+from .jobs import *

netbox/core/models/data.py

@@ -5,7 +5,7 @@ from fnmatch import fnmatchcase
 from urllib.parse import urlparse
 
 from django.conf import settings
-from django.contrib.contenttypes.models import ContentType
+from django.contrib.contenttypes.fields import GenericRelation
 from django.core.exceptions import ValidationError
 from django.core.validators import RegexValidator
 from django.db import models
@@ -14,15 +14,15 @@ from django.utils import timezone
 from django.utils.module_loading import import_string
 from django.utils.translation import gettext as _
 
-from extras.models import JobResult
 from netbox.models import PrimaryModel
-from netbox.models.features import ChangeLoggingMixin
+from netbox.models.features import JobsMixin
 from netbox.registry import registry
 from utilities.files import sha256_hash
 from utilities.querysets import RestrictedQuerySet
 from ..choices import *
 from ..exceptions import SyncError
 from ..signals import post_sync, pre_sync
+from .jobs import Job
 
 __all__ = (
     'DataFile',
@@ -32,7 +32,7 @@ __all__ = (
 logger = logging.getLogger('netbox.core.data')
 
 
-class DataSource(PrimaryModel):
+class DataSource(JobsMixin, PrimaryModel):
     """
     A remote source, such as a git repository, from which DataFiles are synchronized.
     """
@@ -116,17 +116,15 @@ class DataSource(PrimaryModel):
         """
         # Set the status to "syncing"
         self.status = DataSourceStatusChoices.QUEUED
+        DataSource.objects.filter(pk=self.pk).update(status=self.status)
 
         # Enqueue a sync job
-        job_result = JobResult.enqueue_job(
+        return Job.enqueue(
             import_string('core.jobs.sync_datasource'),
-            name=self.name,
-            obj_type=ContentType.objects.get_for_model(DataSource),
-            user=request.user,
+            instance=self,
+            user=request.user
         )
 
-        return job_result
-
     def get_backend(self):
         backend_cls = registry['data_backends'].get(self.type)
         backend_params = self.parameters or {}
@@ -137,8 +135,8 @@ class DataSource(PrimaryModel):
         """
         Create/update/delete child DataFiles as necessary to synchronize with the remote source.
         """
-        if not self.ready_for_sync:
-            raise SyncError(f"Cannot initiate sync; data source not ready/enabled")
+        if self.status == DataSourceStatusChoices.SYNCING:
+            raise SyncError(f"Cannot initiate sync; syncing already in progress.")
 
         # Emit the pre_sync signal
         pre_sync.send(sender=self.__class__, instance=self)
@@ -314,3 +312,14 @@ class DataFile(models.Model):
                 self.data = f.read()
 
         return is_modified
+
+    def write_to_disk(self, path, overwrite=False):
+        """
+        Write the object's data to disk at the specified path
+        """
+        # Check whether file already exists
+        if os.path.isfile(path) and not overwrite:
+            raise FileExistsError()
+
+        with open(path, 'wb+') as new_file:
+            new_file.write(self.data)

netbox/core/models/files.py

@@ -0,0 +1,88 @@
+import logging
+import os
+
+from django.conf import settings
+from django.db import models
+from django.urls import reverse
+from django.utils.translation import gettext as _
+
+from ..choices import ManagedFileRootPathChoices
+from netbox.models.features import SyncedDataMixin
+from utilities.querysets import RestrictedQuerySet
+
+__all__ = (
+    'ManagedFile',
+)
+
+logger = logging.getLogger('netbox.core.files')
+
+
+class ManagedFile(SyncedDataMixin, models.Model):
+    """
+    Database representation for a file on disk. This class is typically wrapped by a proxy class (e.g. ScriptModule)
+    to provide additional functionality.
+    """
+    created = models.DateTimeField(
+        auto_now_add=True
+    )
+    last_updated = models.DateTimeField(
+        editable=False,
+        blank=True,
+        null=True
+    )
+    file_root = models.CharField(
+        max_length=1000,
+        choices=ManagedFileRootPathChoices
+    )
+    file_path = models.FilePathField(
+        editable=False,
+        help_text=_("File path relative to the designated root path")
+    )
+
+    objects = RestrictedQuerySet.as_manager()
+
+    class Meta:
+        ordering = ('file_root', 'file_path')
+        constraints = (
+            models.UniqueConstraint(
+                fields=('file_root', 'file_path'),
+                name='%(app_label)s_%(class)s_unique_root_path'
+            ),
+        )
+        indexes = [
+            models.Index(fields=('file_root', 'file_path'), name='core_managedfile_root_path'),
+        ]
+
+    def __str__(self):
+        return self.name
+
+    def get_absolute_url(self):
+        return reverse('core:managedfile', args=[self.pk])
+
+    @property
+    def name(self):
+        return self.file_path
+
+    @property
+    def full_path(self):
+        return os.path.join(self._resolve_root_path(), self.file_path)
+
+    def _resolve_root_path(self):
+        return {
+            'scripts': settings.SCRIPTS_ROOT,
+            'reports': settings.REPORTS_ROOT,
+        }[self.file_root]
+
+    def sync_data(self):
+        if self.data_file:
+            self.file_path = os.path.basename(self.data_path)
+            self.data_file.write_to_disk(self.full_path, overwrite=True)
+
+    def delete(self, *args, **kwargs):
+        # Delete file from disk
+        try:
+            os.remove(self.full_path)
+        except FileNotFoundError:
+            pass
+
+        return super().delete(*args, **kwargs)

netbox/core/models/jobs.py

@@ -0,0 +1,222 @@
+import uuid
+
+import django_rq
+from django.contrib.auth.models import User
+from django.contrib.contenttypes.fields import GenericForeignKey
+from django.contrib.contenttypes.models import ContentType
+from django.core.validators import MinValueValidator
+from django.db import models
+from django.urls import reverse
+from django.utils import timezone
+from django.utils.translation import gettext as _
+
+from core.choices import JobStatusChoices
+from extras.constants import EVENT_JOB_END, EVENT_JOB_START
+from extras.utils import FeatureQuery
+from netbox.config import get_config
+from netbox.constants import RQ_QUEUE_DEFAULT
+from utilities.querysets import RestrictedQuerySet
+from utilities.rqworker import get_queue_for_model
+
+__all__ = (
+    'Job',
+)
+
+
+class Job(models.Model):
+    """
+    Tracks the lifecycle of a job which represents a background task (e.g. the execution of a custom script).
+    """
+    object_type = models.ForeignKey(
+        to=ContentType,
+        related_name='jobs',
+        limit_choices_to=FeatureQuery('jobs'),
+        on_delete=models.CASCADE,
+    )
+    object_id = models.PositiveBigIntegerField(
+        blank=True,
+        null=True
+    )
+    object = GenericForeignKey(
+        ct_field='object_type',
+        fk_field='object_id',
+        for_concrete_model=False
+    )
+    name = models.CharField(
+        max_length=200
+    )
+    created = models.DateTimeField(
+        auto_now_add=True
+    )
+    scheduled = models.DateTimeField(
+        null=True,
+        blank=True
+    )
+    interval = models.PositiveIntegerField(
+        blank=True,
+        null=True,
+        validators=(
+            MinValueValidator(1),
+        ),
+        help_text=_("Recurrence interval (in minutes)")
+    )
+    started = models.DateTimeField(
+        null=True,
+        blank=True
+    )
+    completed = models.DateTimeField(
+        null=True,
+        blank=True
+    )
+    user = models.ForeignKey(
+        to=User,
+        on_delete=models.SET_NULL,
+        related_name='+',
+        blank=True,
+        null=True
+    )
+    status = models.CharField(
+        max_length=30,
+        choices=JobStatusChoices,
+        default=JobStatusChoices.STATUS_PENDING
+    )
+    data = models.JSONField(
+        null=True,
+        blank=True
+    )
+    job_id = models.UUIDField(
+        unique=True
+    )
+
+    objects = RestrictedQuerySet.as_manager()
+
+    class Meta:
+        ordering = ['-created']
+
+    def __str__(self):
+        return str(self.job_id)
+
+    def get_absolute_url(self):
+        # TODO: Employ dynamic registration
+        if self.object_type.model == 'reportmodule':
+            return reverse(f'extras:report_result', kwargs={'job_pk': self.pk})
+        if self.object_type.model == 'scriptmodule':
+            return reverse(f'extras:script_result', kwargs={'job_pk': self.pk})
+
+    def get_status_color(self):
+        return JobStatusChoices.colors.get(self.status)
+
+    @property
+    def duration(self):
+        if not self.completed:
+            return None
+
+        start_time = self.started or self.created
+
+        if not start_time:
+            return None
+
+        duration = self.completed - start_time
+        minutes, seconds = divmod(duration.total_seconds(), 60)
+
+        return f"{int(minutes)} minutes, {seconds:.2f} seconds"
+
+    def delete(self, *args, **kwargs):
+        super().delete(*args, **kwargs)
+
+        rq_queue_name = get_config().QUEUE_MAPPINGS.get(self.object_type.model, RQ_QUEUE_DEFAULT)
+        queue = django_rq.get_queue(rq_queue_name)
+        job = queue.fetch_job(str(self.job_id))
+
+        if job:
+            job.cancel()
+
+    def start(self):
+        """
+        Record the job's start time and update its status to "running."
+        """
+        if self.started is not None:
+            return
+
+        # Start the job
+        self.started = timezone.now()
+        self.status = JobStatusChoices.STATUS_RUNNING
+        self.save()
+
+        # Handle webhooks
+        self.trigger_webhooks(event=EVENT_JOB_START)
+
+    def terminate(self, status=JobStatusChoices.STATUS_COMPLETED):
+        """
+        Mark the job as completed, optionally specifying a particular termination status.
+        """
+        valid_statuses = JobStatusChoices.TERMINAL_STATE_CHOICES
+        if status not in valid_statuses:
+            raise ValueError(f"Invalid status for job termination. Choices are: {', '.join(valid_statuses)}")
+
+        # Mark the job as completed
+        self.status = status
+        self.completed = timezone.now()
+        self.save()
+
+        # Handle webhooks
+        self.trigger_webhooks(event=EVENT_JOB_END)
+
+    @classmethod
+    def enqueue(cls, func, instance, name='', user=None, schedule_at=None, interval=None, **kwargs):
+        """
+        Create a Job instance and enqueue a job using the given callable
+
+        Args:
+            func: The callable object to be enqueued for execution
+            instance: The NetBox object to which this job pertains
+            name: Name for the job (optional)
+            user: The user responsible for running the job
+            schedule_at: Schedule the job to be executed at the passed date and time
+            interval: Recurrence interval (in minutes)
+        """
+        object_type = ContentType.objects.get_for_model(instance, for_concrete_model=False)
+        rq_queue_name = get_queue_for_model(object_type.model)
+        queue = django_rq.get_queue(rq_queue_name)
+        status = JobStatusChoices.STATUS_SCHEDULED if schedule_at else JobStatusChoices.STATUS_PENDING
+        job = Job.objects.create(
+            object_type=object_type,
+            object_id=instance.pk,
+            name=name,
+            status=status,
+            scheduled=schedule_at,
+            interval=interval,
+            user=user,
+            job_id=uuid.uuid4()
+        )
+
+        if schedule_at:
+            queue.enqueue_at(schedule_at, func, job_id=str(job.job_id), job=job, **kwargs)
+        else:
+            queue.enqueue(func, job_id=str(job.job_id), job=job, **kwargs)
+
+        return job
+
+    def trigger_webhooks(self, event):
+        from extras.models import Webhook
+
+        rq_queue_name = get_config().QUEUE_MAPPINGS.get('webhook', RQ_QUEUE_DEFAULT)
+        rq_queue = django_rq.get_queue(rq_queue_name, is_async=False)
+
+        # Fetch any webhooks matching this object type and action
+        webhooks = Webhook.objects.filter(
+            **{f'type_{event}': True},
+            content_types=self.object_type,
+            enabled=True
+        )
+
+        for webhook in webhooks:
+            rq_queue.enqueue(
+                "extras.webhooks_worker.process_webhook",
+                webhook=webhook,
+                model_name=self.object_type.model,
+                event=event,
+                data=self.data,
+                timestamp=str(timezone.now()),
+                username=self.user.username
+            )

netbox/core/tables/__init__.py

@@ -1 +1,2 @@
 from .data import *
+from .jobs import *

netbox/core/tables/jobs.py

@@ -0,0 +1,39 @@
+import django_tables2 as tables
+from django.utils.translation import gettext as _
+
+from netbox.tables import NetBoxTable, columns
+from ..models import Job
+
+
+class JobTable(NetBoxTable):
+    id = tables.Column(
+        linkify=True
+    )
+    name = tables.Column(
+        linkify=True
+    )
+    object_type = columns.ContentTypeColumn(
+        verbose_name=_('Type')
+    )
+    object = tables.Column(
+        linkify=True
+    )
+    status = columns.ChoiceFieldColumn()
+    created = columns.DateTimeColumn()
+    scheduled = columns.DateTimeColumn()
+    interval = columns.DurationColumn()
+    started = columns.DateTimeColumn()
+    completed = columns.DateTimeColumn()
+    actions = columns.ActionsColumn(
+        actions=('delete',)
+    )
+
+    class Meta(NetBoxTable.Meta):
+        model = Job
+        fields = (
+            'pk', 'id', 'object_type', 'object', 'name', 'status', 'created', 'scheduled', 'interval', 'started',
+            'completed', 'user', 'job_id',
+        )
+        default_columns = (
+            'pk', 'id', 'object_type', 'object', 'name', 'status', 'created', 'started', 'completed', 'user',
+        )

netbox/core/urls.py

@@ -19,4 +19,9 @@ urlpatterns = (
     path('data-files/delete/', views.DataFileBulkDeleteView.as_view(), name='datafile_bulk_delete'),
     path('data-files/<int:pk>/', include(get_model_urls('core', 'datafile'))),
 
+    # Job results
+    path('jobs/', views.JobListView.as_view(), name='job_list'),
+    path('jobs/delete/', views.JobBulkDeleteView.as_view(), name='job_bulk_delete'),
+    path('jobs/<int:pk>/delete/', views.JobDeleteView.as_view(), name='job_delete'),
+
 )

netbox/core/views.py

@@ -3,6 +3,7 @@ from django.shortcuts import get_object_or_404, redirect
 
 from netbox.views import generic
 from netbox.views.generic.base import BaseObjectView
+from utilities.rqworker import get_queue_for_model, get_workers_for_queue
 from utilities.utils import count_related
 from utilities.views import register_model_view
 from . import filtersets, forms, tables
@@ -50,9 +51,9 @@ class DataSourceSyncView(BaseObjectView):
 
     def post(self, request, pk):
         datasource = get_object_or_404(self.queryset, pk=pk)
-        job_result = datasource.enqueue_sync_job(request)
+        job = datasource.enqueue_sync_job(request)
 
-        messages.success(request, f"Queued job #{job_result.pk} to sync {datasource}")
+        messages.success(request, f"Queued job #{job.pk} to sync {datasource}")
         return redirect(datasource.get_absolute_url())
 
 
@@ -70,7 +71,6 @@ class DataSourceDeleteView(generic.ObjectDeleteView):
 class DataSourceBulkImportView(generic.BulkImportView):
     queryset = DataSource.objects.all()
     model_form = forms.DataSourceImportForm
-    table = tables.DataSourceTable
 
 
 class DataSourceBulkEditView(generic.BulkEditView):
@@ -116,3 +116,25 @@ class DataFileBulkDeleteView(generic.BulkDeleteView):
     queryset = DataFile.objects.defer('data')
     filterset = filtersets.DataFileFilterSet
     table = tables.DataFileTable
+
+
+#
+# Jobs
+#
+
+class JobListView(generic.ObjectListView):
+    queryset = Job.objects.all()
+    filterset = filtersets.JobFilterSet
+    filterset_form = forms.JobFilterForm
+    table = tables.JobTable
+    actions = ('export', 'delete', 'bulk_delete', )
+
+
+class JobDeleteView(generic.ObjectDeleteView):
+    queryset = Job.objects.all()
+
+
+class JobBulkDeleteView(generic.BulkDeleteView):
+    queryset = Job.objects.all()
+    filterset = filtersets.JobFilterSet
+    table = tables.JobTable

netbox/dcim/api/serializers.py

@@ -482,7 +482,10 @@ class InterfaceTemplateSerializer(ValidatedModelSerializer):
         default=None
     )
     type = ChoiceField(choices=InterfaceTypeChoices)
-    bridge = NestedInterfaceTemplateSerializer(required=False, allow_null=True)
+    bridge = NestedInterfaceTemplateSerializer(
+        required=False,
+        allow_null=True
+    )
     poe_mode = ChoiceField(
         choices=InterfacePoEModeChoices,
         required=False,
@@ -499,8 +502,8 @@ class InterfaceTemplateSerializer(ValidatedModelSerializer):
     class Meta:
         model = InterfaceTemplate
         fields = [
-            'id', 'url', 'display', 'device_type', 'module_type', 'name', 'label', 'type', 'bridge', 'enabled', 'mgmt_only', 'description',
-            'poe_mode', 'poe_type', 'created', 'last_updated',
+            'id', 'url', 'display', 'device_type', 'module_type', 'name', 'label', 'type', 'enabled', 'mgmt_only',
+            'description', 'bridge', 'poe_mode', 'poe_type', 'created', 'last_updated',
         ]
 
 
@@ -610,6 +613,7 @@ class InventoryItemTemplateSerializer(ValidatedModelSerializer):
 
 class DeviceRoleSerializer(NetBoxModelSerializer):
     url = serializers.HyperlinkedIdentityField(view_name='dcim-api:devicerole-detail')
+    config_template = NestedConfigTemplateSerializer(required=False, allow_null=True, default=None)
     device_count = serializers.IntegerField(read_only=True)
     virtualmachine_count = serializers.IntegerField(read_only=True)
 
@@ -624,6 +628,7 @@ class DeviceRoleSerializer(NetBoxModelSerializer):
 class PlatformSerializer(NetBoxModelSerializer):
     url = serializers.HyperlinkedIdentityField(view_name='dcim-api:platform-detail')
     manufacturer = NestedManufacturerSerializer(required=False, allow_null=True)
+    config_template = NestedConfigTemplateSerializer(required=False, allow_null=True, default=None)
     device_count = serializers.IntegerField(read_only=True)
     virtualmachine_count = serializers.IntegerField(read_only=True)
 

netbox/dcim/api/views.py

@@ -1,11 +1,11 @@
-import socket
-
-from django.http import Http404, HttpResponse, HttpResponseForbidden
+from django.http import Http404, HttpResponse
 from django.shortcuts import get_object_or_404
 from drf_spectacular.utils import extend_schema, extend_schema_view, OpenApiParameter
 from drf_spectacular.types import OpenApiTypes
 from rest_framework.decorators import action
+from rest_framework.renderers import JSONRenderer
 from rest_framework.response import Response
+from rest_framework.status import HTTP_400_BAD_REQUEST
 from rest_framework.routers import APIRootView
 from rest_framework.viewsets import ViewSet
 
@@ -14,14 +14,14 @@ from dcim import filtersets
 from dcim.constants import CABLE_TRACE_SVG_DEFAULT_WIDTH
 from dcim.models import *
 from dcim.svg import CableTraceSVG
-from extras.api.views import ConfigContextQuerySetMixin
+from extras.api.nested_serializers import NestedConfigTemplateSerializer
+from extras.api.mixins import ConfigContextQuerySetMixin, ConfigTemplateRenderMixin
 from ipam.models import Prefix, VLAN
 from netbox.api.authentication import IsAuthenticatedOrLoginNotRequired
-from netbox.api.exceptions import ServiceUnavailable
 from netbox.api.metadata import ContentTypeMetadata
 from netbox.api.pagination import StripCountAnnotationsPaginator
+from netbox.api.renderers import TextRenderer
 from netbox.api.viewsets import NetBoxModelViewSet
-from netbox.config import get_config
 from netbox.constants import NESTED_SERIALIZER_PREFIX
 from utilities.api import get_serializer_for_model
 from utilities.utils import count_related
@@ -275,7 +275,7 @@ class ManufacturerViewSet(NetBoxModelViewSet):
 #
 
 class DeviceTypeViewSet(NetBoxModelViewSet):
-    queryset = DeviceType.objects.prefetch_related('manufacturer', 'tags').annotate(
+    queryset = DeviceType.objects.prefetch_related('manufacturer', 'default_platform', 'tags').annotate(
         device_count=count_related(Device, 'device_type')
     )
     serializer_class = serializers.DeviceTypeSerializer
@@ -386,10 +386,10 @@ class PlatformViewSet(NetBoxModelViewSet):
 # Devices/modules
 #
 
-class DeviceViewSet(ConfigContextQuerySetMixin, NetBoxModelViewSet):
+class DeviceViewSet(ConfigContextQuerySetMixin, ConfigTemplateRenderMixin, NetBoxModelViewSet):
     queryset = Device.objects.prefetch_related(
         'device_type__manufacturer', 'device_role', 'tenant', 'platform', 'site', 'location', 'rack', 'parent_bay',
-        'virtual_chassis__master', 'primary_ip4__nat_outside', 'primary_ip6__nat_outside', 'tags',
+        'virtual_chassis__master', 'primary_ip4__nat_outside', 'primary_ip6__nat_outside', 'config_template', 'tags',
     )
     filterset_class = filtersets.DeviceFilterSet
     pagination_class = StripCountAnnotationsPaginator
@@ -414,6 +414,19 @@ class DeviceViewSet(ConfigContextQuerySetMixin, NetBoxModelViewSet):
 
         return serializers.DeviceWithConfigContextSerializer
 
+    @action(detail=True, methods=['post'], url_path='render-config', renderer_classes=[JSONRenderer, TextRenderer])
+    def render_config(self, request, pk):
+        """
+        Resolve and render the preferred ConfigTemplate for this Device.
+        """
+        device = self.get_object()
+        configtemplate = device.get_config_template()
+        if not configtemplate:
+            return Response({'error': 'No config template found for this device.'}, status=HTTP_400_BAD_REQUEST)
+        context = {**request.data, 'device': device}
+
+        return self.render_configtemplate(request, configtemplate, context)
+
 
 class VirtualDeviceContextViewSet(NetBoxModelViewSet):
     queryset = VirtualDeviceContext.objects.prefetch_related(

netbox/dcim/choices.py

@@ -1137,7 +1137,6 @@ class InterfacePoETypeChoices(ChoiceSet):
 
     TYPE_1_8023AF = 'type1-ieee802.3af'
     TYPE_2_8023AT = 'type2-ieee802.3at'
-    TYPE_2_8023AZ = 'type2-ieee802.3az'
     TYPE_3_8023BT = 'type3-ieee802.3bt'
     TYPE_4_8023BT = 'type4-ieee802.3bt'
 
@@ -1152,7 +1151,6 @@ class InterfacePoETypeChoices(ChoiceSet):
             (
                 (TYPE_1_8023AF, '802.3af (Type 1)'),
                 (TYPE_2_8023AT, '802.3at (Type 2)'),
-                (TYPE_2_8023AZ, '802.3az (Type 2)'),
                 (TYPE_3_8023BT, '802.3bt (Type 3)'),
                 (TYPE_4_8023BT, '802.3bt (Type 4)'),
             )

netbox/dcim/filtersets.py

@@ -685,6 +685,10 @@ class InterfaceTemplateFilterSet(ChangeLoggedModelFilterSet, ModularDeviceTypeCo
         choices=InterfaceTypeChoices,
         null_value=None
     )
+    bridge_id = django_filters.ModelMultipleChoiceFilter(
+        field_name='bridge',
+        queryset=InterfaceTemplate.objects.all()
+    )
     poe_mode = django_filters.MultipleChoiceFilter(
         choices=InterfacePoEModeChoices
     )

netbox/dcim/forms/bulk_edit.py

@@ -1164,6 +1164,14 @@ class InterfaceBulkEditForm(
         },
         label=_('LAG')
     )
+    vdcs = DynamicModelMultipleChoiceField(
+        queryset=VirtualDeviceContext.objects.all(),
+        required=False,
+        label='Virtual Device Contexts',
+        query_params={
+            'device_id': '$device',
+        }
+    )
     speed = forms.IntegerField(
         required=False,
         widget=SelectSpeedWidget(),
@@ -1226,14 +1234,14 @@ class InterfaceBulkEditForm(
     fieldsets = (
         (None, ('module', 'type', 'label', 'speed', 'duplex', 'description')),
         ('Addressing', ('vrf', 'mac_address', 'wwn')),
-        ('Operation', ('mtu', 'tx_power', 'enabled', 'mgmt_only', 'mark_connected')),
+        ('Operation', ('vdcs', 'mtu', 'tx_power', 'enabled', 'mgmt_only', 'mark_connected')),
         ('PoE', ('poe_mode', 'poe_type')),
         ('Related Interfaces', ('parent', 'bridge', 'lag')),
         ('802.1Q Switching', ('mode', 'vlan_group', 'untagged_vlan', 'tagged_vlans')),
         ('Wireless', ('rf_role', 'rf_channel', 'rf_channel_frequency', 'rf_channel_width')),
     )
     nullable_fields = (
-        'module', 'label', 'parent', 'bridge', 'lag', 'speed', 'duplex', 'mac_address', 'wwn', 'mtu', 'description',
+        'module', 'label', 'parent', 'bridge', 'lag', 'speed', 'duplex', 'mac_address', 'wwn', 'vdcs', 'mtu', 'description',
         'poe_mode', 'poe_type', 'mode', 'rf_channel', 'rf_channel_frequency', 'rf_channel_width', 'tx_power',
         'vlan_group', 'untagged_vlan', 'tagged_vlans', 'vrf',
     )

netbox/dcim/forms/bulk_import.py

@@ -12,7 +12,9 @@ from extras.models import ConfigTemplate
 from ipam.models import VRF
 from netbox.forms import NetBoxModelImportForm
 from tenancy.models import Tenant
-from utilities.forms import CSVChoiceField, CSVContentTypeField, CSVModelChoiceField, CSVTypedChoiceField, SlugField
+from utilities.forms import (
+    CSVChoiceField, CSVContentTypeField, CSVModelChoiceField, CSVTypedChoiceField, SlugField, CSVModelMultipleChoiceField
+)
 from virtualization.models import Cluster
 from wireless.choices import WirelessRoleChoices
 from .common import ModuleCommonForm
@@ -280,12 +282,14 @@ class ManufacturerImportForm(NetBoxModelImportForm):
 class DeviceTypeImportForm(NetBoxModelImportForm):
     manufacturer = forms.ModelChoiceField(
         queryset=Manufacturer.objects.all(),
-        to_field_name='name'
+        to_field_name='name',
+        help_text=_('The manufacturer which produces this device type')
     )
     default_platform = forms.ModelChoiceField(
         queryset=Platform.objects.all(),
         to_field_name='name',
         required=False,
+        help_text=_('The default platform for devices of this type (optional)')
     )
 
     class Meta:
@@ -689,6 +693,12 @@ class InterfaceImportForm(NetBoxModelImportForm):
         to_field_name='name',
         help_text=_('Parent LAG interface')
     )
+    vdcs = CSVModelMultipleChoiceField(
+        queryset=VirtualDeviceContext.objects.all(),
+        required=False,
+        to_field_name='name',
+        help_text='VDC names separated by commas, encased with double quotes (e.g. "vdc1, vdc2, vdc3")'
+    )
     type = CSVChoiceField(
         choices=InterfaceTypeChoices,
         help_text=_('Physical medium')
@@ -728,7 +738,7 @@ class InterfaceImportForm(NetBoxModelImportForm):
         model = Interface
         fields = (
             'device', 'name', 'label', 'parent', 'bridge', 'lag', 'type', 'speed', 'duplex', 'enabled',
-            'mark_connected', 'mac_address', 'wwn', 'mtu', 'mgmt_only', 'description', 'poe_mode', 'poe_type', 'mode',
+            'mark_connected', 'mac_address', 'wwn', 'vdcs', 'mtu', 'mgmt_only', 'description', 'poe_mode', 'poe_type', 'mode',
             'vrf', 'rf_role', 'rf_channel', 'rf_channel_frequency', 'rf_channel_width', 'tx_power', 'tags'
         )
 
@@ -744,6 +754,7 @@ class InterfaceImportForm(NetBoxModelImportForm):
                 self.fields['parent'].queryset = self.fields['parent'].queryset.filter(**params)
                 self.fields['bridge'].queryset = self.fields['bridge'].queryset.filter(**params)
                 self.fields['lag'].queryset = self.fields['lag'].queryset.filter(**params)
+                self.fields['vdcs'].queryset = self.fields['vdcs'].queryset.filter(**params)
 
     def clean_enabled(self):
         # Make sure enabled is True when it's not included in the uploaded data
@@ -752,6 +763,12 @@ class InterfaceImportForm(NetBoxModelImportForm):
         else:
             return self.cleaned_data['enabled']
 
+    def clean_vdcs(self):
+        for vdc in self.cleaned_data['vdcs']:
+            if vdc.device != self.cleaned_data['device']:
+                raise forms.ValidationError(f"VDC {vdc} is not assigned to device {self.cleaned_data['device']}")
+        return self.cleaned_data['vdcs']
+
 
 class FrontPortImportForm(NetBoxModelImportForm):
     device = CSVModelChoiceField(

netbox/dcim/forms/model_forms.py

@@ -283,7 +283,7 @@ class DeviceTypeForm(NetBoxModelForm):
     comments = CommentField()
 
     fieldsets = (
-        ('Device Type', ('manufacturer', 'model', 'slug', 'description', 'tags', 'default_platform')),
+        ('Device Type', ('manufacturer', 'model', 'slug', 'default_platform', 'description', 'tags')),
         ('Chassis', (
             'u_height', 'is_full_depth', 'part_number', 'subdevice_role', 'airflow', 'weight', 'weight_unit',
         )),
@@ -293,8 +293,9 @@ class DeviceTypeForm(NetBoxModelForm):
     class Meta:
         model = DeviceType
         fields = [
-            'manufacturer', 'model', 'slug', 'part_number', 'u_height', 'is_full_depth', 'subdevice_role', 'airflow',
-            'weight', 'weight_unit', 'front_image', 'rear_image', 'description', 'comments', 'tags', 'default_platform'
+            'manufacturer', 'model', 'slug', 'default_platform', 'part_number', 'u_height', 'is_full_depth',
+            'subdevice_role', 'airflow', 'weight', 'weight_unit', 'front_image', 'rear_image', 'description',
+            'comments', 'tags',
         ]
         widgets = {
             'front_image': ClearableFileInput(attrs={

netbox/dcim/forms/object_create.py

@@ -359,7 +359,7 @@ class VirtualChassisCreateForm(NetBoxModelForm):
     class Meta:
         model = VirtualChassis
         fields = [
-            'name', 'domain', 'region', 'site_group', 'site', 'rack', 'members', 'initial_position', 'tags',
+            'name', 'domain', 'description', 'region', 'site_group', 'site', 'rack', 'members', 'initial_position', 'tags',
         ]
 
     def clean(self):

netbox/dcim/graphql/schema.py

@@ -2,126 +2,248 @@ import graphene
 
 from netbox.graphql.fields import ObjectField, ObjectListField
 from .types import *
+from dcim import models
 from .types import VirtualDeviceContextType
+from utilities.graphql_optimizer import gql_query_optimizer
 
 
 class DCIMQuery(graphene.ObjectType):
     cable = ObjectField(CableType)
     cable_list = ObjectListField(CableType)
 
+    def resolve_cable_list(root, info, **kwargs):
+        return gql_query_optimizer(models.Cable.objects.all(), info)
+
     console_port = ObjectField(ConsolePortType)
     console_port_list = ObjectListField(ConsolePortType)
 
+    def resolve_console_port_list(root, info, **kwargs):
+        return gql_query_optimizer(models.ConsolePort.objects.all(), info)
+
     console_port_template = ObjectField(ConsolePortTemplateType)
     console_port_template_list = ObjectListField(ConsolePortTemplateType)
 
+    def resolve_console_port_template_list(root, info, **kwargs):
+        return gql_query_optimizer(models.ConsolePortTemplate.objects.all(), info)
+
     console_server_port = ObjectField(ConsoleServerPortType)
     console_server_port_list = ObjectListField(ConsoleServerPortType)
 
+    def resolve_console_server_port_list(root, info, **kwargs):
+        return gql_query_optimizer(models.ConsoleServerPort.objects.all(), info)
+
     console_server_port_template = ObjectField(ConsoleServerPortTemplateType)
     console_server_port_template_list = ObjectListField(ConsoleServerPortTemplateType)
 
+    def resolve_console_server_port_template_list(root, info, **kwargs):
+        return gql_query_optimizer(models.ConsoleServerPortTemplate.objects.all(), info)
+
     device = ObjectField(DeviceType)
     device_list = ObjectListField(DeviceType)
 
+    def resolve_device_list(root, info, **kwargs):
+        return gql_query_optimizer(models.Device.objects.all(), info)
+
     device_bay = ObjectField(DeviceBayType)
     device_bay_list = ObjectListField(DeviceBayType)
 
+    def resolve_device_bay_list(root, info, **kwargs):
+        return gql_query_optimizer(models.DeviceBay.objects.all(), info)
+
     device_bay_template = ObjectField(DeviceBayTemplateType)
     device_bay_template_list = ObjectListField(DeviceBayTemplateType)
 
+    def resolve_device_bay_template_list(root, info, **kwargs):
+        return gql_query_optimizer(models.DeviceBayTemplate.objects.all(), info)
+
     device_role = ObjectField(DeviceRoleType)
     device_role_list = ObjectListField(DeviceRoleType)
 
+    def resolve_device_role_list(root, info, **kwargs):
+        return gql_query_optimizer(models.DeviceRole.objects.all(), info)
+
     device_type = ObjectField(DeviceTypeType)
     device_type_list = ObjectListField(DeviceTypeType)
 
+    def resolve_device_type_list(root, info, **kwargs):
+        return gql_query_optimizer(models.DeviceType.objects.all(), info)
+
     front_port = ObjectField(FrontPortType)
     front_port_list = ObjectListField(FrontPortType)
 
+    def resolve_front_port_list(root, info, **kwargs):
+        return gql_query_optimizer(models.FrontPort.objects.all(), info)
+
     front_port_template = ObjectField(FrontPortTemplateType)
     front_port_template_list = ObjectListField(FrontPortTemplateType)
 
+    def resolve_front_port_template_list(root, info, **kwargs):
+        return gql_query_optimizer(models.FrontPortTemplate.objects.all(), info)
+
     interface = ObjectField(InterfaceType)
     interface_list = ObjectListField(InterfaceType)
 
+    def resolve_interface_list(root, info, **kwargs):
+        return gql_query_optimizer(models.Interface.objects.all(), info)
+
     interface_template = ObjectField(InterfaceTemplateType)
     interface_template_list = ObjectListField(InterfaceTemplateType)
 
+    def resolve_interface_template_list(root, info, **kwargs):
+        return gql_query_optimizer(models.InterfaceTemplate.objects.all(), info)
+
     inventory_item = ObjectField(InventoryItemType)
     inventory_item_list = ObjectListField(InventoryItemType)
 
+    def resolve_inventory_item_list(root, info, **kwargs):
+        return gql_query_optimizer(models.InventoryItem.objects.all(), info)
+
     inventory_item_role = ObjectField(InventoryItemRoleType)
     inventory_item_role_list = ObjectListField(InventoryItemRoleType)
 
+    def resolve_inventory_item_role_list(root, info, **kwargs):
+        return gql_query_optimizer(models.InventoryItemRole.objects.all(), info)
+
     inventory_item_template = ObjectField(InventoryItemTemplateType)
     inventory_item_template_list = ObjectListField(InventoryItemTemplateType)
 
+    def resolve_inventory_item_template_list(root, info, **kwargs):
+        return gql_query_optimizer(models.InventoryItemTemplate.objects.all(), info)
+
     location = ObjectField(LocationType)
     location_list = ObjectListField(LocationType)
 
+    def resolve_location_list(root, info, **kwargs):
+        return gql_query_optimizer(models.Location.objects.all(), info)
+
     manufacturer = ObjectField(ManufacturerType)
     manufacturer_list = ObjectListField(ManufacturerType)
 
+    def resolve_manufacturer_list(root, info, **kwargs):
+        return gql_query_optimizer(models.Manufacturer.objects.all(), info)
+
     module = ObjectField(ModuleType)
     module_list = ObjectListField(ModuleType)
 
+    def resolve_module_list(root, info, **kwargs):
+        return gql_query_optimizer(models.Module.objects.all(), info)
+
     module_bay = ObjectField(ModuleBayType)
     module_bay_list = ObjectListField(ModuleBayType)
 
+    def resolve_module_bay_list(root, info, **kwargs):
+        return gql_query_optimizer(models.ModuleBay.objects.all(), info)
+
     module_bay_template = ObjectField(ModuleBayTemplateType)
     module_bay_template_list = ObjectListField(ModuleBayTemplateType)
 
+    def resolve_module_bay_template_list(root, info, **kwargs):
+        return gql_query_optimizer(models.ModuleBayTemplate.objects.all(), info)
+
     module_type = ObjectField(ModuleTypeType)
     module_type_list = ObjectListField(ModuleTypeType)
 
+    def resolve_module_type_list(root, info, **kwargs):
+        return gql_query_optimizer(models.ModuleType.objects.all(), info)
+
     platform = ObjectField(PlatformType)
     platform_list = ObjectListField(PlatformType)
 
+    def resolve_platform_list(root, info, **kwargs):
+        return gql_query_optimizer(models.Platform.objects.all(), info)
+
     power_feed = ObjectField(PowerFeedType)
     power_feed_list = ObjectListField(PowerFeedType)
 
+    def resolve_power_feed_list(root, info, **kwargs):
+        return gql_query_optimizer(models.PowerFeed.objects.all(), info)
+
     power_outlet = ObjectField(PowerOutletType)
     power_outlet_list = ObjectListField(PowerOutletType)
 
+    def resolve_power_outlet_list(root, info, **kwargs):
+        return gql_query_optimizer(models.PowerOutlet.objects.all(), info)
+
     power_outlet_template = ObjectField(PowerOutletTemplateType)
     power_outlet_template_list = ObjectListField(PowerOutletTemplateType)
 
+    def resolve_power_outlet_template_list(root, info, **kwargs):
+        return gql_query_optimizer(models.PowerOutletTemplate.objects.all(), info)
+
     power_panel = ObjectField(PowerPanelType)
     power_panel_list = ObjectListField(PowerPanelType)
 
+    def resolve_power_panel_list(root, info, **kwargs):
+        return gql_query_optimizer(models.PowerPanel.objects.all(), info)
+
     power_port = ObjectField(PowerPortType)
     power_port_list = ObjectListField(PowerPortType)
 
+    def resolve_power_port_list(root, info, **kwargs):
+        return gql_query_optimizer(models.PowerPort.objects.all(), info)
+
     power_port_template = ObjectField(PowerPortTemplateType)
     power_port_template_list = ObjectListField(PowerPortTemplateType)
 
+    def resolve_power_port_template_list(root, info, **kwargs):
+        return gql_query_optimizer(models.PowerPortTemplate.objects.all(), info)
+
     rack = ObjectField(RackType)
     rack_list = ObjectListField(RackType)
 
+    def resolve_rack_list(root, info, **kwargs):
+        return gql_query_optimizer(models.Rack.objects.all(), info)
+
     rack_reservation = ObjectField(RackReservationType)
     rack_reservation_list = ObjectListField(RackReservationType)
 
+    def resolve_rack_reservation_list(root, info, **kwargs):
+        return gql_query_optimizer(models.RackReservation.objects.all(), info)
+
     rack_role = ObjectField(RackRoleType)
     rack_role_list = ObjectListField(RackRoleType)
 
+    def resolve_rack_role_list(root, info, **kwargs):
+        return gql_query_optimizer(models.RackRole.objects.all(), info)
+
     rear_port = ObjectField(RearPortType)
     rear_port_list = ObjectListField(RearPortType)
 
+    def resolve_rear_port_list(root, info, **kwargs):
+        return gql_query_optimizer(models.RearPort.objects.all(), info)
+
     rear_port_template = ObjectField(RearPortTemplateType)
     rear_port_template_list = ObjectListField(RearPortTemplateType)
 
+    def resolve_rear_port_template_list(root, info, **kwargs):
+        return gql_query_optimizer(models.RearPortTemplate.objects.all(), info)
+
     region = ObjectField(RegionType)
     region_list = ObjectListField(RegionType)
 
+    def resolve_region_list(root, info, **kwargs):
+        return gql_query_optimizer(models.Region.objects.all(), info)
+
     site = ObjectField(SiteType)
     site_list = ObjectListField(SiteType)
 
+    def resolve_site_list(root, info, **kwargs):
+        return gql_query_optimizer(models.Site.objects.all(), info)
+
     site_group = ObjectField(SiteGroupType)
     site_group_list = ObjectListField(SiteGroupType)
 
+    def resolve_site_group_list(root, info, **kwargs):
+        return gql_query_optimizer(models.SiteGroup.objects.all(), info)
+
     virtual_chassis = ObjectField(VirtualChassisType)
     virtual_chassis_list = ObjectListField(VirtualChassisType)
 
+    def resolve_virtual_chassis_list(root, info, **kwargs):
+        return gql_query_optimizer(models.VirtualChassis.objects.all(), info)
+
     virtual_device_context = ObjectField(VirtualDeviceContextType)
     virtual_device_context_list = ObjectListField(VirtualDeviceContextType)
+
+    def resolve_virtual_device_context_list(root, info, **kwargs):
+        return gql_query_optimizer(models.VirtualDeviceContext.objects.all(), info)

netbox/dcim/models/device_component_templates.py

@@ -377,6 +377,8 @@ class InterfaceTemplate(ModularComponentTemplateModel):
         super().clean()
 
         if self.bridge:
+            if self.pk and self.bridge_id == self.pk:
+                raise ValidationError({'bridge': "An interface cannot be bridged to itself."})
             if self.device_type and self.device_type != self.bridge.device_type:
                 raise ValidationError({
                     'bridge': f"Bridge interface ({self.bridge}) must belong to the same device type"

netbox/dcim/tests/test_filtersets.py

@@ -1142,11 +1142,36 @@ class InterfaceTemplateTestCase(TestCase, ChangeLoggedFilterSetTests):
         )
         DeviceType.objects.bulk_create(device_types)
 
-        InterfaceTemplate.objects.bulk_create((
-            InterfaceTemplate(device_type=device_types[0], name='Interface 1', type=InterfaceTypeChoices.TYPE_1GE_FIXED, enabled=True, mgmt_only=True, poe_mode=InterfacePoEModeChoices.MODE_PD, poe_type=InterfacePoETypeChoices.TYPE_1_8023AF),
-            InterfaceTemplate(device_type=device_types[1], name='Interface 2', type=InterfaceTypeChoices.TYPE_1GE_GBIC, enabled=False, mgmt_only=False, poe_mode=InterfacePoEModeChoices.MODE_PSE, poe_type=InterfacePoETypeChoices.TYPE_2_8023AT),
-            InterfaceTemplate(device_type=device_types[2], name='Interface 3', type=InterfaceTypeChoices.TYPE_1GE_SFP, mgmt_only=False),
-        ))
+        interface_templates = (
+            InterfaceTemplate(
+                device_type=device_types[0],
+                name='Interface 1',
+                type=InterfaceTypeChoices.TYPE_1GE_FIXED,
+                enabled=True,
+                mgmt_only=True,
+                poe_mode=InterfacePoEModeChoices.MODE_PD,
+                poe_type=InterfacePoETypeChoices.TYPE_1_8023AF
+            ),
+            InterfaceTemplate(
+                device_type=device_types[1],
+                name='Interface 2',
+                type=InterfaceTypeChoices.TYPE_1GE_GBIC,
+                enabled=False,
+                mgmt_only=False,
+                poe_mode=InterfacePoEModeChoices.MODE_PSE,
+                poe_type=InterfacePoETypeChoices.TYPE_2_8023AT
+            ),
+            InterfaceTemplate(
+                device_type=device_types[2],
+                name='Interface 3',
+                type=InterfaceTypeChoices.TYPE_1GE_SFP,
+                mgmt_only=False
+            ),
+        )
+        InterfaceTemplate.objects.bulk_create(interface_templates)
+        interface_templates[0].bridge = interface_templates[1]
+        interface_templates[1].bridge = interface_templates[0]
+        InterfaceTemplate.objects.bulk_update(interface_templates, ['bridge'])
 
     def test_name(self):
         params = {'name': ['Interface 1', 'Interface 2']}
@@ -1173,6 +1198,10 @@ class InterfaceTemplateTestCase(TestCase, ChangeLoggedFilterSetTests):
         params = {'mgmt_only': 'false'}
         self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
 
+    def test_bridge(self):
+        params = {'bridge_id': [InterfaceTemplate.objects.filter(bridge__isnull=False).first().bridge_id]}
+        self.assertEqual(self.filterset(params, self.queryset).qs.count(), 1)
+
     def test_poe_mode(self):
         params = {'poe_mode': [InterfacePoEModeChoices.MODE_PD, InterfacePoEModeChoices.MODE_PSE]}
         self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)

netbox/dcim/tests/test_views.py

@@ -1994,7 +1994,7 @@ class ModuleTestCase(
         }
 
         initial_count = Module.objects.count()
-        self.assertHttpStatus(self.client.post(**request), 200)
+        self.assertHttpStatus(self.client.post(**request), 302)
         self.assertEqual(Module.objects.count(), initial_count + len(csv_data) - 1)
         self.assertEqual(Interface.objects.filter(device=device).count(), 0)
 
@@ -2010,7 +2010,7 @@ class ModuleTestCase(
         }
 
         initial_count = Module.objects.count()
-        self.assertHttpStatus(self.client.post(**request), 200)
+        self.assertHttpStatus(self.client.post(**request), 302)
         self.assertEqual(Module.objects.count(), initial_count + len(csv_data) - 1)
         self.assertEqual(Interface.objects.filter(device=device).count(), 5)
 
@@ -2086,7 +2086,7 @@ class ModuleTestCase(
         }
 
         initial_count = self._get_queryset().count()
-        self.assertHttpStatus(self.client.post(**request), 200)
+        self.assertHttpStatus(self.client.post(**request), 302)
         self.assertEqual(self._get_queryset().count(), initial_count + len(csv_data) - 1)
 
         # Re-retrieve interface to get new module id

netbox/dcim/views.py

@@ -27,7 +27,6 @@ from utilities.views import GetReturnURLMixin, ObjectPermissionRequiredMixin, Vi
 from virtualization.models import VirtualMachine
 from . import filtersets, forms, tables
 from .choices import DeviceFaceChoices
-from .constants import NONCONNECTABLE_IFACE_TYPES
 from .models import *
 
 CABLE_TERMINATION_TYPES = {
@@ -240,7 +239,6 @@ class RegionDeleteView(generic.ObjectDeleteView):
 class RegionBulkImportView(generic.BulkImportView):
     queryset = Region.objects.all()
     model_form = forms.RegionImportForm
-    table = tables.RegionTable
 
 
 class RegionBulkEditView(generic.BulkEditView):
@@ -316,7 +314,6 @@ class SiteGroupDeleteView(generic.ObjectDeleteView):
 class SiteGroupBulkImportView(generic.BulkImportView):
     queryset = SiteGroup.objects.all()
     model_form = forms.SiteGroupImportForm
-    table = tables.SiteGroupTable
 
 
 class SiteGroupBulkEditView(generic.BulkEditView):
@@ -422,7 +419,6 @@ class SiteDeleteView(generic.ObjectDeleteView):
 class SiteBulkImportView(generic.BulkImportView):
     queryset = Site.objects.all()
     model_form = forms.SiteImportForm
-    table = tables.SiteTable
 
 
 class SiteBulkEditView(generic.BulkEditView):
@@ -499,7 +495,6 @@ class LocationDeleteView(generic.ObjectDeleteView):
 class LocationBulkImportView(generic.BulkImportView):
     queryset = Location.objects.all()
     model_form = forms.LocationImportForm
-    table = tables.LocationTable
 
 
 class LocationBulkEditView(generic.BulkEditView):
@@ -568,7 +563,6 @@ class RackRoleDeleteView(generic.ObjectDeleteView):
 class RackRoleBulkImportView(generic.BulkImportView):
     queryset = RackRole.objects.all()
     model_form = forms.RackRoleImportForm
-    table = tables.RackRoleTable
 
 
 class RackRoleBulkEditView(generic.BulkEditView):
@@ -728,7 +722,6 @@ class RackDeleteView(generic.ObjectDeleteView):
 class RackBulkImportView(generic.BulkImportView):
     queryset = Rack.objects.all()
     model_form = forms.RackImportForm
-    table = tables.RackTable
 
 
 class RackBulkEditView(generic.BulkEditView):
@@ -781,7 +774,6 @@ class RackReservationDeleteView(generic.ObjectDeleteView):
 class RackReservationImportView(generic.BulkImportView):
     queryset = RackReservation.objects.all()
     model_form = forms.RackReservationImportForm
-    table = tables.RackReservationTable
 
     def save_object(self, object_form, request):
         """
@@ -854,7 +846,6 @@ class ManufacturerDeleteView(generic.ObjectDeleteView):
 class ManufacturerBulkImportView(generic.BulkImportView):
     queryset = Manufacturer.objects.all()
     model_form = forms.ManufacturerImportForm
-    table = tables.ManufacturerTable
 
 
 class ManufacturerBulkEditView(generic.BulkEditView):
@@ -1083,7 +1074,6 @@ class DeviceTypeImportView(generic.BulkImportView):
     ]
     queryset = DeviceType.objects.all()
     model_form = forms.DeviceTypeImportForm
-    table = tables.DeviceTypeTable
     related_object_forms = {
         'console-ports': forms.ConsolePortTemplateImportForm,
         'console-server-ports': forms.ConsoleServerPortTemplateImportForm,
@@ -1275,7 +1265,6 @@ class ModuleTypeImportView(generic.BulkImportView):
     ]
     queryset = ModuleType.objects.all()
     model_form = forms.ModuleTypeImportForm
-    table = tables.ModuleTypeTable
     related_object_forms = {
         'console-ports': forms.ConsolePortTemplateImportForm,
         'console-server-ports': forms.ConsoleServerPortTemplateImportForm,
@@ -1722,7 +1711,6 @@ class DeviceRoleDeleteView(generic.ObjectDeleteView):
 class DeviceRoleBulkImportView(generic.BulkImportView):
     queryset = DeviceRole.objects.all()
     model_form = forms.DeviceRoleImportForm
-    table = tables.DeviceRoleTable
 
 
 class DeviceRoleBulkEditView(generic.BulkEditView):
@@ -1786,7 +1774,6 @@ class PlatformDeleteView(generic.ObjectDeleteView):
 class PlatformBulkImportView(generic.BulkImportView):
     queryset = Platform.objects.all()
     model_form = forms.PlatformImportForm
-    table = tables.PlatformTable
 
 
 class PlatformBulkEditView(generic.BulkEditView):
@@ -2000,6 +1987,17 @@ class DeviceInventoryView(DeviceComponentsView):
     )
 
 
+@register_model_view(Device, 'configcontext', path='config-context')
+class DeviceConfigContextView(ObjectConfigContextView):
+    queryset = Device.objects.annotate_config_context_data()
+    base_template = 'dcim/device/base.html'
+    tab = ViewTab(
+        label=_('Config Context'),
+        permission='extras.view_configcontext',
+        weight=2000
+    )
+
+
 @register_model_view(Device, 'render-config')
 class DeviceRenderConfigView(generic.ObjectView):
     queryset = Device.objects.all()
@@ -2007,7 +2005,7 @@ class DeviceRenderConfigView(generic.ObjectView):
     tab = ViewTab(
         label=_('Render Config'),
         permission='extras.view_configtemplate',
-        weight=2000
+        weight=2100
     )
 
     def get_extra_context(self, request, instance):
@@ -2033,21 +2031,9 @@ class DeviceRenderConfigView(generic.ObjectView):
         }
 
 
-@register_model_view(Device, 'configcontext', path='config-context')
-class DeviceConfigContextView(ObjectConfigContextView):
-    queryset = Device.objects.annotate_config_context_data()
-    base_template = 'dcim/device/base.html'
-    tab = ViewTab(
-        label=_('Config Context'),
-        permission='extras.view_configcontext',
-        weight=2100
-    )
-
-
 class DeviceBulkImportView(generic.BulkImportView):
     queryset = Device.objects.all()
     model_form = forms.DeviceImportForm
-    table = tables.DeviceImportTable
 
     def save_object(self, object_form, request):
         obj = object_form.save()
@@ -2125,7 +2111,6 @@ class ModuleDeleteView(generic.ObjectDeleteView):
 class ModuleBulkImportView(generic.BulkImportView):
     queryset = Module.objects.all()
     model_form = forms.ModuleImportForm
-    table = tables.ModuleTable
 
 
 class ModuleBulkEditView(generic.BulkEditView):
@@ -2178,7 +2163,6 @@ class ConsolePortDeleteView(generic.ObjectDeleteView):
 class ConsolePortBulkImportView(generic.BulkImportView):
     queryset = ConsolePort.objects.all()
     model_form = forms.ConsolePortImportForm
-    table = tables.ConsolePortTable
 
 
 class ConsolePortBulkEditView(generic.BulkEditView):
@@ -2243,7 +2227,6 @@ class ConsoleServerPortDeleteView(generic.ObjectDeleteView):
 class ConsoleServerPortBulkImportView(generic.BulkImportView):
     queryset = ConsoleServerPort.objects.all()
     model_form = forms.ConsoleServerPortImportForm
-    table = tables.ConsoleServerPortTable
 
 
 class ConsoleServerPortBulkEditView(generic.BulkEditView):
@@ -2308,7 +2291,6 @@ class PowerPortDeleteView(generic.ObjectDeleteView):
 class PowerPortBulkImportView(generic.BulkImportView):
     queryset = PowerPort.objects.all()
     model_form = forms.PowerPortImportForm
-    table = tables.PowerPortTable
 
 
 class PowerPortBulkEditView(generic.BulkEditView):
@@ -2373,7 +2355,6 @@ class PowerOutletDeleteView(generic.ObjectDeleteView):
 class PowerOutletBulkImportView(generic.BulkImportView):
     queryset = PowerOutlet.objects.all()
     model_form = forms.PowerOutletImportForm
-    table = tables.PowerOutletTable
 
 
 class PowerOutletBulkEditView(generic.BulkEditView):
@@ -2484,7 +2465,6 @@ class InterfaceDeleteView(generic.ObjectDeleteView):
 class InterfaceBulkImportView(generic.BulkImportView):
     queryset = Interface.objects.all()
     model_form = forms.InterfaceImportForm
-    table = tables.InterfaceTable
 
 
 class InterfaceBulkEditView(generic.BulkEditView):
@@ -2549,7 +2529,6 @@ class FrontPortDeleteView(generic.ObjectDeleteView):
 class FrontPortBulkImportView(generic.BulkImportView):
     queryset = FrontPort.objects.all()
     model_form = forms.FrontPortImportForm
-    table = tables.FrontPortTable
 
 
 class FrontPortBulkEditView(generic.BulkEditView):
@@ -2614,7 +2593,6 @@ class RearPortDeleteView(generic.ObjectDeleteView):
 class RearPortBulkImportView(generic.BulkImportView):
     queryset = RearPort.objects.all()
     model_form = forms.RearPortImportForm
-    table = tables.RearPortTable
 
 
 class RearPortBulkEditView(generic.BulkEditView):
@@ -2679,7 +2657,6 @@ class ModuleBayDeleteView(generic.ObjectDeleteView):
 class ModuleBayBulkImportView(generic.BulkImportView):
     queryset = ModuleBay.objects.all()
     model_form = forms.ModuleBayImportForm
-    table = tables.ModuleBayTable
 
 
 class ModuleBayBulkEditView(generic.BulkEditView):
@@ -2805,7 +2782,6 @@ class DeviceBayDepopulateView(generic.ObjectEditView):
 class DeviceBayBulkImportView(generic.BulkImportView):
     queryset = DeviceBay.objects.all()
     model_form = forms.DeviceBayImportForm
-    table = tables.DeviceBayTable
 
 
 class DeviceBayBulkEditView(generic.BulkEditView):
@@ -2864,7 +2840,6 @@ class InventoryItemDeleteView(generic.ObjectDeleteView):
 class InventoryItemBulkImportView(generic.BulkImportView):
     queryset = InventoryItem.objects.all()
     model_form = forms.InventoryItemImportForm
-    table = tables.InventoryItemTable
 
 
 class InventoryItemBulkEditView(generic.BulkEditView):
@@ -2921,7 +2896,6 @@ class InventoryItemRoleDeleteView(generic.ObjectDeleteView):
 class InventoryItemRoleBulkImportView(generic.BulkImportView):
     queryset = InventoryItemRole.objects.all()
     model_form = forms.InventoryItemRoleImportForm
-    table = tables.InventoryItemRoleTable
 
 
 class InventoryItemRoleBulkEditView(generic.BulkEditView):
@@ -3116,7 +3090,6 @@ class CableDeleteView(generic.ObjectDeleteView):
 class CableBulkImportView(generic.BulkImportView):
     queryset = Cable.objects.all()
     model_form = forms.CableImportForm
-    table = tables.CableTable
 
 
 class CableBulkEditView(generic.BulkEditView):
@@ -3399,7 +3372,6 @@ class VirtualChassisRemoveMemberView(ObjectPermissionRequiredMixin, GetReturnURL
 class VirtualChassisBulkImportView(generic.BulkImportView):
     queryset = VirtualChassis.objects.all()
     model_form = forms.VirtualChassisImportForm
-    table = tables.VirtualChassisTable
 
 
 class VirtualChassisBulkEditView(generic.BulkEditView):
@@ -3456,7 +3428,6 @@ class PowerPanelDeleteView(generic.ObjectDeleteView):
 class PowerPanelBulkImportView(generic.BulkImportView):
     queryset = PowerPanel.objects.all()
     model_form = forms.PowerPanelImportForm
-    table = tables.PowerPanelTable
 
 
 class PowerPanelBulkEditView(generic.BulkEditView):
@@ -3504,7 +3475,6 @@ class PowerFeedDeleteView(generic.ObjectDeleteView):
 class PowerFeedBulkImportView(generic.BulkImportView):
     queryset = PowerFeed.objects.all()
     model_form = forms.PowerFeedImportForm
-    table = tables.PowerFeedTable
 
 
 class PowerFeedBulkEditView(generic.BulkEditView):
@@ -3566,7 +3536,6 @@ class VirtualDeviceContextDeleteView(generic.ObjectDeleteView):
 class VirtualDeviceContextBulkImportView(generic.BulkImportView):
     queryset = VirtualDeviceContext.objects.all()
     model_form = forms.VirtualDeviceContextImportForm
-    table = tables.VirtualDeviceContextTable
 
 
 class VirtualDeviceContextBulkEditView(generic.BulkEditView):

netbox/extras/admin.py

@@ -6,7 +6,7 @@ from django.utils.html import format_html
 
 from netbox.config import get_config, PARAMS
 from .forms import ConfigRevisionForm
-from .models import ConfigRevision, JobResult
+from .models import ConfigRevision
 
 
 @admin.register(ConfigRevision)
@@ -39,7 +39,7 @@ class ConfigRevisionAdmin(admin.ModelAdmin):
             'fields': ('DEFAULT_USER_PREFERENCES',),
         }),
         ('Miscellaneous', {
-            'fields': ('MAINTENANCE_MODE', 'GRAPHQL_ENABLED', 'CHANGELOG_RETENTION', 'JOBRESULT_RETENTION', 'MAPS_URL'),
+            'fields': ('MAINTENANCE_MODE', 'GRAPHQL_ENABLED', 'CHANGELOG_RETENTION', 'JOB_RETENTION', 'MAPS_URL'),
         }),
         ('Config Revision', {
             'fields': ('comment',),

netbox/extras/api/mixins.py

@@ -0,0 +1,46 @@
+from rest_framework.response import Response
+
+from .nested_serializers import NestedConfigTemplateSerializer
+
+__all__ = (
+    'ConfigContextQuerySetMixin',
+)
+
+
+class ConfigContextQuerySetMixin:
+    """
+    Used by views that work with config context models (device and virtual machine).
+    Provides a get_queryset() method which deals with adding the config context
+    data annotation or not.
+    """
+    def get_queryset(self):
+        """
+        Build the proper queryset based on the request context
+
+        If the `brief` query param equates to True or the `exclude` query param
+        includes `config_context` as a value, return the base queryset.
+
+        Else, return the queryset annotated with config context data
+        """
+        queryset = super().get_queryset()
+        request = self.get_serializer_context()['request']
+        if self.brief or 'config_context' in request.query_params.get('exclude', []):
+            return queryset
+        return queryset.annotate_config_context_data()
+
+
+class ConfigTemplateRenderMixin:
+
+    def render_configtemplate(self, request, configtemplate, context):
+        output = configtemplate.render(context=context)
+
+        # If the client has requested "text/plain", return the raw content.
+        if request.accepted_renderer.format == 'txt':
+            return Response(output)
+
+        template_serializer = NestedConfigTemplateSerializer(configtemplate, context={'request': request})
+
+        return Response({
+            'configtemplate': template_serializer.data,
+            'content': output
+        })

netbox/extras/api/nested_serializers.py

@@ -1,9 +1,7 @@
 from rest_framework import serializers
 
-from extras import choices, models
-from netbox.api.fields import ChoiceField
+from extras import models
 from netbox.api.serializers import NestedTagSerializer, WritableNestedSerializer
-from users.api.nested_serializers import NestedUserSerializer
 
 __all__ = [
     'NestedConfigContextSerializer',
@@ -12,7 +10,6 @@ __all__ = [
     'NestedCustomLinkSerializer',
     'NestedExportTemplateSerializer',
     'NestedImageAttachmentSerializer',
-    'NestedJobResultSerializer',
     'NestedJournalEntrySerializer',
     'NestedSavedFilterSerializer',
     'NestedTagSerializer',  # Defined in netbox.api.serializers
@@ -90,15 +87,3 @@ class NestedJournalEntrySerializer(WritableNestedSerializer):
     class Meta:
         model = models.JournalEntry
         fields = ['id', 'url', 'display', 'created']
-
-
-class NestedJobResultSerializer(serializers.ModelSerializer):
-    url = serializers.HyperlinkedIdentityField(view_name='extras-api:jobresult-detail')
-    status = ChoiceField(choices=choices.JobResultStatusChoices)
-    user = NestedUserSerializer(
-        read_only=True
-    )
-
-    class Meta:
-        model = models.JobResult
-        fields = ['url', 'created', 'completed', 'user', 'status']

netbox/extras/api/serializers.py

@@ -3,7 +3,8 @@ from django.contrib.contenttypes.models import ContentType
 from django.core.exceptions import ObjectDoesNotExist
 from rest_framework import serializers
 
-from core.api.nested_serializers import NestedDataSourceSerializer, NestedDataFileSerializer
+from core.api.serializers import JobSerializer
+from core.api.nested_serializers import NestedDataSourceSerializer, NestedDataFileSerializer, NestedJobSerializer
 from dcim.api.nested_serializers import (
     NestedDeviceRoleSerializer, NestedDeviceTypeSerializer, NestedLocationSerializer, NestedPlatformSerializer,
     NestedRegionSerializer, NestedSiteSerializer, NestedSiteGroupSerializer,
@@ -38,7 +39,6 @@ __all__ = (
     'DashboardSerializer',
     'ExportTemplateSerializer',
     'ImageAttachmentSerializer',
-    'JobResultSerializer',
     'JournalEntrySerializer',
     'ObjectChangeSerializer',
     'ReportDetailSerializer',
@@ -411,28 +411,6 @@ class ConfigTemplateSerializer(TaggableModelSerializer, ValidatedModelSerializer
         ]
 
 
-#
-# Job Results
-#
-
-class JobResultSerializer(BaseModelSerializer):
-    url = serializers.HyperlinkedIdentityField(view_name='extras-api:jobresult-detail')
-    user = NestedUserSerializer(
-        read_only=True
-    )
-    status = ChoiceField(choices=JobResultStatusChoices, read_only=True)
-    obj_type = ContentTypeField(
-        read_only=True
-    )
-
-    class Meta:
-        model = JobResult
-        fields = [
-            'id', 'url', 'display', 'status', 'created', 'scheduled', 'interval', 'started', 'completed', 'name',
-            'obj_type', 'user', 'data', 'job_id',
-        ]
-
-
 #
 # Reports
 #
@@ -448,11 +426,11 @@ class ReportSerializer(serializers.Serializer):
     name = serializers.CharField(max_length=255)
     description = serializers.CharField(max_length=255, required=False)
     test_methods = serializers.ListField(child=serializers.CharField(max_length=255))
-    result = NestedJobResultSerializer()
+    result = NestedJobSerializer()
 
 
 class ReportDetailSerializer(ReportSerializer):
-    result = JobResultSerializer()
+    result = JobSerializer()
 
 
 class ReportInputSerializer(serializers.Serializer):
@@ -475,7 +453,7 @@ class ScriptSerializer(serializers.Serializer):
     name = serializers.CharField(read_only=True)
     description = serializers.CharField(read_only=True)
     vars = serializers.SerializerMethodField(read_only=True)
-    result = NestedJobResultSerializer()
+    result = NestedJobSerializer()
 
     @extend_schema_field(serializers.JSONField(allow_null=True))
     def get_vars(self, instance):
@@ -485,7 +463,7 @@ class ScriptSerializer(serializers.Serializer):
 
 
 class ScriptDetailSerializer(ScriptSerializer):
-    result = JobResultSerializer()
+    result = JobSerializer()
 
 
 class ScriptInputSerializer(serializers.Serializer):

netbox/extras/api/urls.py

@@ -20,7 +20,6 @@ router.register('config-templates', views.ConfigTemplateViewSet)
 router.register('reports', views.ReportViewSet, basename='report')
 router.register('scripts', views.ScriptViewSet, basename='script')
 router.register('object-changes', views.ObjectChangeViewSet)
-router.register('job-results', views.JobResultViewSet)
 router.register('content-types', views.ContentTypeViewSet)
 
 app_name = 'extras-api'

netbox/extras/api/views.py

@@ -1,5 +1,6 @@
 from django.contrib.contenttypes.models import ContentType
 from django.http import Http404
+from django.shortcuts import get_object_or_404
 from django_rq.queues import get_connection
 from rest_framework import status
 from rest_framework.decorators import action
@@ -12,12 +13,12 @@ from rest_framework.routers import APIRootView
 from rest_framework.viewsets import ReadOnlyModelViewSet, ViewSet
 from rq import Worker
 
+from core.choices import JobStatusChoices
+from core.models import Job
 from extras import filtersets
-from extras.choices import JobResultStatusChoices
 from extras.models import *
-from extras.models import CustomField
-from extras.reports import get_report, get_reports, run_report
-from extras.scripts import get_script, get_scripts, run_script
+from extras.reports import get_module_and_report, run_report
+from extras.scripts import get_module_and_script, run_script
 from netbox.api.authentication import IsAuthenticatedOrLoginNotRequired
 from netbox.api.features import SyncedDataMixin
 from netbox.api.metadata import ContentTypeMetadata
@@ -26,7 +27,7 @@ from netbox.api.viewsets import NetBoxModelViewSet
 from utilities.exceptions import RQWorkerNotRunningException
 from utilities.utils import copy_safe_request, count_related
 from . import serializers
-from .nested_serializers import NestedConfigTemplateSerializer
+from .mixins import ConfigTemplateRenderMixin
 
 
 class ExtrasRootView(APIRootView):
@@ -37,28 +38,6 @@ class ExtrasRootView(APIRootView):
         return 'Extras'
 
 
-class ConfigContextQuerySetMixin:
-    """
-    Used by views that work with config context models (device and virtual machine).
-    Provides a get_queryset() method which deals with adding the config context
-    data annotation or not.
-    """
-    def get_queryset(self):
-        """
-        Build the proper queryset based on the request context
-
-        If the `brief` query param equates to True or the `exclude` query param
-        includes `config_context` as a value, return the base queryset.
-
-        Else, return the queryset annotated with config context data
-        """
-        queryset = super().get_queryset()
-        request = self.get_serializer_context()['request']
-        if self.brief or 'config_context' in request.query_params.get('exclude', []):
-            return queryset
-        return queryset.annotate_config_context_data()
-
-
 #
 # Webhooks
 #
@@ -165,7 +144,7 @@ class ConfigContextViewSet(SyncedDataMixin, NetBoxModelViewSet):
 # Config templates
 #
 
-class ConfigTemplateViewSet(SyncedDataMixin, NetBoxModelViewSet):
+class ConfigTemplateViewSet(SyncedDataMixin, ConfigTemplateRenderMixin, NetBoxModelViewSet):
     queryset = ConfigTemplate.objects.prefetch_related('data_source', 'data_file')
     serializer_class = serializers.ConfigTemplateSerializer
     filterset_class = filtersets.ConfigTemplateFilterSet
@@ -177,17 +156,9 @@ class ConfigTemplateViewSet(SyncedDataMixin, NetBoxModelViewSet):
         return the raw rendered content, rather than serialized JSON.
         """
         configtemplate = self.get_object()
-        output = configtemplate.render(context=request.data)
+        context = request.data
 
-        # If the client has requested "text/plain", return the raw content.
-        if request.accepted_renderer.format == 'txt':
-            return Response(output)
-
-        template_serializer = NestedConfigTemplateSerializer(configtemplate, context={'request': request})
-        return Response({
-            'configtemplate': template_serializer.data,
-            'content': output
-        })
+        return self.render_configtemplate(request, configtemplate, context)
 
 
 #
@@ -200,41 +171,38 @@ class ReportViewSet(ViewSet):
     schema = None
     lookup_value_regex = '[^/]+'  # Allow dots
 
-    def _retrieve_report(self, pk):
-
-        # Read the PK as "<module>.<report>"
-        if '.' not in pk:
-            raise Http404
+    def _get_report(self, pk):
+        try:
             module_name, report_name = pk.split('.', maxsplit=1)
+        except ValueError:
+            raise Http404
 
-        # Raise a 404 on an invalid Report module/name
-        report = get_report(module_name, report_name)
+        module, report = get_module_and_report(module_name, report_name)
         if report is None:
             raise Http404
 
-        return report
+        return module, report
 
     def list(self, request):
         """
         Compile all reports and their related results (if any). Result data is deferred in the list view.
         """
-        report_list = []
         report_content_type = ContentType.objects.get(app_label='extras', model='report')
         results = {
             r.name: r
-            for r in JobResult.objects.filter(
-                obj_type=report_content_type,
-                status__in=JobResultStatusChoices.TERMINAL_STATE_CHOICES
+            for r in Job.objects.filter(
+                object_type=report_content_type,
+                status__in=JobStatusChoices.TERMINAL_STATE_CHOICES
             ).order_by('name', '-created').distinct('name').defer('data')
         }
 
-        # Iterate through all available Reports.
-        for module_name, reports in get_reports().items():
-            for report in reports.values():
+        report_list = []
+        for report_module in ReportModule.objects.restrict(request.user):
+            report_list.extend([report() for report in report_module.reports.values()])
 
-                # Attach the relevant JobResult (if any) to each Report.
+        # Attach Job objects to each report (if any)
+        for report in report_list:
             report.result = results.get(report.full_name, None)
-                report_list.append(report)
 
         serializer = serializers.ReportSerializer(report_list, many=True, context={
             'request': request,
@@ -246,14 +214,14 @@ class ReportViewSet(ViewSet):
         """
         Retrieve a single Report identified as "<module>.<report>".
         """
+        module, report = self._get_report(pk)
 
-        # Retrieve the Report and JobResult, if any.
-        report = self._retrieve_report(pk)
-        report_content_type = ContentType.objects.get(app_label='extras', model='report')
-        report.result = JobResult.objects.filter(
-            obj_type=report_content_type,
-            name=report.full_name,
-            status__in=JobResultStatusChoices.TERMINAL_STATE_CHOICES
+        # Retrieve the Report and Job, if any.
+        object_type = ContentType.objects.get(app_label='extras', model='reportmodule')
+        report.result = Job.objects.filter(
+            object_type=object_type,
+            name=report.name,
+            status__in=JobStatusChoices.TERMINAL_STATE_CHOICES
         ).first()
 
         serializer = serializers.ReportDetailSerializer(report, context={
@@ -265,7 +233,7 @@ class ReportViewSet(ViewSet):
     @action(detail=True, methods=['post'])
     def run(self, request, pk):
         """
-        Run a Report identified as "<module>.<script>" and return the pending JobResult as the result
+        Run a Report identified as "<module>.<script>" and return the pending Job as the result
         """
         # Check that the user has permission to run reports.
         if not request.user.has_perm('extras.run_report'):
@@ -275,22 +243,20 @@ class ReportViewSet(ViewSet):
         if not Worker.count(get_connection('default')):
             raise RQWorkerNotRunningException()
 
-        # Retrieve and run the Report. This will create a new JobResult.
-        report = self._retrieve_report(pk)
+        # Retrieve and run the Report. This will create a new Job.
+        module, report = self._get_report(pk)
         input_serializer = serializers.ReportInputSerializer(data=request.data)
 
         if input_serializer.is_valid():
-            job_result = JobResult.enqueue_job(
+            report.result = Job.enqueue(
                 run_report,
-                name=report.full_name,
-                obj_type=ContentType.objects.get_for_model(Report),
+                instance=module,
+                name=report.class_name,
                 user=request.user,
                 job_timeout=report.job_timeout,
                 schedule_at=input_serializer.validated_data.get('schedule_at'),
                 interval=input_serializer.validated_data.get('interval')
             )
-            report.result = job_result
-
             serializer = serializers.ReportDetailSerializer(report, context={'request': request})
 
             return Response(serializer.data)
@@ -308,42 +274,47 @@ class ScriptViewSet(ViewSet):
     lookup_value_regex = '[^/]+'  # Allow dots
 
     def _get_script(self, pk):
+        try:
             module_name, script_name = pk.split('.', maxsplit=1)
-        script = get_script(module_name, script_name)
+        except ValueError:
+            raise Http404
+
+        module, script = get_module_and_script(module_name, script_name)
         if script is None:
             raise Http404
-        return script
+
+        return module, script
 
     def list(self, request):
 
         script_content_type = ContentType.objects.get(app_label='extras', model='script')
         results = {
             r.name: r
-            for r in JobResult.objects.filter(
-                obj_type=script_content_type,
-                status__in=JobResultStatusChoices.TERMINAL_STATE_CHOICES
+            for r in Job.objects.filter(
+                object_type=script_content_type,
+                status__in=JobStatusChoices.TERMINAL_STATE_CHOICES
             ).order_by('name', '-created').distinct('name').defer('data')
         }
 
-        flat_list = []
-        for script_list in get_scripts().values():
-            flat_list.extend(script_list.values())
+        script_list = []
+        for script_module in ScriptModule.objects.restrict(request.user):
+            script_list.extend(script_module.scripts.values())
 
-        # Attach JobResult objects to each script (if any)
-        for script in flat_list:
+        # Attach Job objects to each script (if any)
+        for script in script_list:
             script.result = results.get(script.full_name, None)
 
-        serializer = serializers.ScriptSerializer(flat_list, many=True, context={'request': request})
+        serializer = serializers.ScriptSerializer(script_list, many=True, context={'request': request})
 
         return Response(serializer.data)
 
     def retrieve(self, request, pk):
-        script = self._get_script(pk)
-        script_content_type = ContentType.objects.get(app_label='extras', model='script')
-        script.result = JobResult.objects.filter(
-            obj_type=script_content_type,
-            name=script.full_name,
-            status__in=JobResultStatusChoices.TERMINAL_STATE_CHOICES
+        module, script = self._get_script(pk)
+        object_type = ContentType.objects.get(app_label='extras', model='scriptmodule')
+        script.result = Job.objects.filter(
+            object_type=object_type,
+            name=script.name,
+            status__in=JobStatusChoices.TERMINAL_STATE_CHOICES
         ).first()
         serializer = serializers.ScriptDetailSerializer(script, context={'request': request})
 
@@ -351,13 +322,13 @@ class ScriptViewSet(ViewSet):
 
     def post(self, request, pk):
         """
-        Run a Script identified as "<module>.<script>" and return the pending JobResult as the result
+        Run a Script identified as "<module>.<script>" and return the pending Job as the result
         """
 
         if not request.user.has_perm('extras.run_script'):
             raise PermissionDenied("This user does not have permission to run scripts.")
 
-        script = self._get_script(pk)()
+        module, script = self._get_script(pk)
         input_serializer = serializers.ScriptInputSerializer(data=request.data)
 
         # Check that at least one RQ worker is running
@@ -365,10 +336,10 @@ class ScriptViewSet(ViewSet):
             raise RQWorkerNotRunningException()
 
         if input_serializer.is_valid():
-            job_result = JobResult.enqueue_job(
+            script.result = Job.enqueue(
                 run_script,
-                name=script.full_name,
-                obj_type=ContentType.objects.get_for_model(Script),
+                instance=module,
+                name=script.class_name,
                 user=request.user,
                 data=input_serializer.data['data'],
                 request=copy_safe_request(request),
@@ -377,7 +348,6 @@ class ScriptViewSet(ViewSet):
                 schedule_at=input_serializer.validated_data.get('schedule_at'),
                 interval=input_serializer.validated_data.get('interval')
             )
-            script.result = job_result
             serializer = serializers.ScriptDetailSerializer(script, context={'request': request})
 
             return Response(serializer.data)
@@ -399,19 +369,6 @@ class ObjectChangeViewSet(ReadOnlyModelViewSet):
     filterset_class = filtersets.ObjectChangeFilterSet
 
 
-#
-# Job Results
-#
-
-class JobResultViewSet(ReadOnlyModelViewSet):
-    """
-    Retrieve a list of job results
-    """
-    queryset = JobResult.objects.prefetch_related('user')
-    serializer_class = serializers.JobResultSerializer
-    filterset_class = filtersets.JobResultFilterSet
-
-
 #
 # ContentTypes
 #