diff --git a/docs/customization/custom-fields.md b/docs/customization/custom-fields.md index 1f9a4a8bf..4658cc7e6 100644 --- a/docs/customization/custom-fields.md +++ b/docs/customization/custom-fields.md @@ -74,6 +74,8 @@ If a default value is specified for a selection field, it must exactly match one An object or multi-object custom field can be used to refer to a particular NetBox object or objects as the "value" for a custom field. These custom fields must define an `object_type`, which determines the type of object to which custom field instances point. +By default, an object choice field will make all objects of that type available for selection in the drop-down. The list choices can be filtered to show only objects with certain values by providing a `query_params` dict in the Related Object Filter field, as a JSON value. More information about `query_params` can be found [here](./custom-scripts.md#objectvar). + ## Custom Fields in Templates Several features within NetBox, such as export templates and webhooks, utilize Jinja2 templating. For convenience, objects which support custom field assignment expose custom field data through the `cf` property. This is a bit cleaner than accessing custom field data through the actual field (`custom_field_data`). diff --git a/docs/customization/custom-validation.md b/docs/customization/custom-validation.md index 909846e20..4a2aab998 100644 --- a/docs/customization/custom-validation.md +++ b/docs/customization/custom-validation.md @@ -86,8 +86,6 @@ CUSTOM_VALIDATORS = { #### Referencing Related Object Attributes -!!! info "This feature was introduced in NetBox v4.0." - The attributes of a related object can be referenced by specifying a dotted path. For example, to reference the name of a region to which a site is assigned, use `region.name`: ```python @@ -104,8 +102,6 @@ CUSTOM_VALIDATORS = { #### Validating Request Parameters -!!! info "This feature was introduced in NetBox v4.0." - In addition to validating object attributes, custom validators can also match against parameters of the current request (where available). For example, the following rule will permit only the user named "admin" to modify an object: ```json diff --git a/docs/development/models.md b/docs/development/models.md index 19b7be6de..a762409b6 100644 --- a/docs/development/models.md +++ b/docs/development/models.md @@ -18,7 +18,7 @@ Depending on its classification, each NetBox model may support various features | [Custom links](../customization/custom-links.md) | `CustomLinksMixin` | `custom_links` | These models support the assignment of custom links | | [Custom validation](../customization/custom-validation.md) | `CustomValidationMixin` | - | Supports the enforcement of custom validation rules | | [Export templates](../customization/export-templates.md) | `ExportTemplatesMixin` | `export_templates` | Users can create custom export templates for these models | -| [Job results](../features/background-jobs.md) | `JobsMixin` | `jobs` | Users can create custom export templates for these models | +| [Job results](../features/background-jobs.md) | `JobsMixin` | `jobs` | Background jobs can be scheduled for these models | | [Journaling](../features/journaling.md) | `JournalingMixin` | `journaling` | These models support persistent historical commentary | | [Synchronized data](../integrations/synchronized-data.md) | `SyncedDataMixin` | `synced_data` | Certain model data can be automatically synchronized from a remote data source | | [Tagging](../models/extras/tag.md) | `TagsMixin` | `tags` | The models can be tagged with user-defined tags | diff --git a/docs/features/event-rules.md b/docs/features/event-rules.md index 158dc111a..14b54f000 100644 --- a/docs/features/event-rules.md +++ b/docs/features/event-rules.md @@ -1,9 +1,10 @@ # Event Rules -NetBox includes the ability to execute certain functions in response to internal object changes. These include: +NetBox includes the ability to automatically perform certain functions in response to internal events. These include: -* [Scripts](../customization/custom-scripts.md) execution -* [Webhooks](../integrations/webhooks.md) execution +* Executing a [custom script](../customization/custom-scripts.md) +* Sending a [webhook](../integrations/webhooks.md) +* Generating [user notifications](../features/notifications.md) For example, suppose you want to automatically configure a monitoring system to start monitoring a device when its operational status is changed to active, and remove it from monitoring for any other status. You can create a webhook in NetBox for the device model and craft its content and destination URL to effect the desired change on the receiving system. You can then associate an event rule with this webhook and the webhook will be sent automatically by NetBox whenever the configured constraints are met. diff --git a/docs/features/notifications.md b/docs/features/notifications.md new file mode 100644 index 000000000..a28a17947 --- /dev/null +++ b/docs/features/notifications.md @@ -0,0 +1,10 @@ +# Notifications + +!!! info "This feature was introduced in NetBox v4.1." + +NetBox includes a system for generating user notifications, which can be marked as read or deleted by individual users. There are two built-in mechanisms for generating a notification: + +* A user can subscribe to an object. When that object is modified, a notification is created to inform the user of the change. +* An [event rule](./event-rules.md) can be defined to automatically generate a notification for one or more users in response to specific system events. + +Additionally, NetBox plugins can generate notifications for their own purposes. diff --git a/docs/models/circuits/circuitgroup.md b/docs/models/circuits/circuitgroup.md index 6d1503509..faa9dbc14 100644 --- a/docs/models/circuits/circuitgroup.md +++ b/docs/models/circuits/circuitgroup.md @@ -1,5 +1,7 @@ # Circuit Groups +!!! info "This feature was introduced in NetBox v4.1." + [Circuits](./circuit.md) can be arranged into administrative groups for organization. The assignment of a circuit to a group is optional. ## Fields diff --git a/docs/models/dcim/moduletype.md b/docs/models/dcim/moduletype.md index 7077e16c2..225873d61 100644 --- a/docs/models/dcim/moduletype.md +++ b/docs/models/dcim/moduletype.md @@ -42,4 +42,6 @@ The numeric weight of the module, including a unit designation (e.g. 3 kilograms ### Airflow +!!! info "The `airflow` field was introduced in NetBox v4.1." + The direction in which air circulates through the device chassis for cooling. diff --git a/docs/models/dcim/racktype.md b/docs/models/dcim/racktype.md index 476dbe178..73fb08bcd 100644 --- a/docs/models/dcim/racktype.md +++ b/docs/models/dcim/racktype.md @@ -1,5 +1,7 @@ # Rack Types +!!! info "This feature was introduced in NetBox v4.1." + A rack type defines the physical characteristics of a particular model of [rack](./rack.md). ## Fields diff --git a/docs/models/extras/customfield.md b/docs/models/extras/customfield.md index 2353bc2b9..626f320be 100644 --- a/docs/models/extras/customfield.md +++ b/docs/models/extras/customfield.md @@ -42,6 +42,15 @@ The type of data this field holds. This must be one of the following: For object and multiple-object fields only. Designates the type of NetBox object being referenced. +### Related Object Filter + +!!! info "This field was introduced in NetBox v4.1." + +For object and multi-object custom fields, a filter may be defined to limit the available objects when populating a field value. This filter maps object attributes to values. For example, `{"status": "active"}` will include only objects with a status of "active." + +!!! warning + This setting is employed for convenience only, and should not be relied upon to enforce data integrity. + ### Weight A numeric weight used to override alphabetic ordering of fields by name. Custom fields with a lower weight will be listed before those with a higher weight. (Note that weight applies within the context of a custom field group, if defined.) diff --git a/docs/models/extras/eventrule.md b/docs/models/extras/eventrule.md index c105a2630..b48e17a1e 100644 --- a/docs/models/extras/eventrule.md +++ b/docs/models/extras/eventrule.md @@ -18,17 +18,22 @@ The type(s) of object in NetBox that will trigger the rule. If not selected, the event rule will not be processed. -### Events +### Events Types -The events which will trigger the rule. At least one event type must be selected. +The event types which will trigger the rule. At least one event type must be selected. -| Name | Description | -|------------|--------------------------------------| -| Creations | A new object has been created | -| Updates | An existing object has been modified | -| Deletions | An object has been deleted | -| Job starts | A job for an object starts | -| Job ends | A job for an object terminates | +| Name | Description | +|----------------|---------------------------------------------| +| Object created | A new object has been created | +| Object updated | An existing object has been modified | +| Object deleted | An object has been deleted | +| Job started | A background job is initiated | +| Job completed | A background job completes successfully | +| Job failed | A background job fails | +| Job errored | A background job is aborted due to an error | + +!!! tip "Custom Event Types" + The above list includes only built-in event types. NetBox plugins can also register their own custom event types. ### Conditions diff --git a/docs/models/ipam/vlangroup.md b/docs/models/ipam/vlangroup.md index 67050ab4c..20989452f 100644 --- a/docs/models/ipam/vlangroup.md +++ b/docs/models/ipam/vlangroup.md @@ -16,6 +16,8 @@ A unique URL-friendly identifier. (This value can be used for filtering.) ### VLAN ID Ranges +!!! info "This field replaced the legacy `min_vid` and `max_vid` fields in NetBox v4.1." + The set of VLAN IDs which are encompassed by the group. By default, this will be the entire range of valid IEEE 802.1Q VLAN IDs (1 to 4094, inclusive). VLANs created within a group must have a VID that falls within one of these ranges. Ranges may not overlap. ### Scope diff --git a/docs/models/virtualization/virtualmachine.md b/docs/models/virtualization/virtualmachine.md index 7a801ca65..7ea31111c 100644 --- a/docs/models/virtualization/virtualmachine.md +++ b/docs/models/virtualization/virtualmachine.md @@ -50,9 +50,13 @@ The amount of running memory provisioned, in megabytes. ### Disk -The amount of disk storage provisioned, in gigabytes. +The amount of disk storage provisioned, in megabytes. + +!!! warning + This field may be directly modified only on virtual machines which do not define discrete [virtual disks](./virtualdisk.md). Otherwise, it will report the sum of all attached disks. ### Serial Number -Optional serial number assigned to this VM. +!!! info "This field was introduced in NetBox v4.1." +Optional serial number assigned to this virtual machine. Unlike devices, uniqueness is not enforced for virtual machine serial numbers. diff --git a/docs/models/wireless/wirelesslink.md b/docs/models/wireless/wirelesslink.md index e670b69ec..7553902b0 100644 --- a/docs/models/wireless/wirelesslink.md +++ b/docs/models/wireless/wirelesslink.md @@ -20,6 +20,12 @@ The operational status of the link. Options include: The service set identifier (SSID) for the wireless link (optional). +### Distance + +!!! info "This field was introduced in NetBox v4.1." + +The distance between the link's two endpoints, including a unit designation (e.g. 100 meters or 25 feet). + ### Authentication Type The type of wireless authentication in use. Options include: @@ -40,7 +46,3 @@ The security cipher used to apply wireless authentication. Options include: ### Pre-Shared Key The security key configured on each client to grant access to the secured wireless LAN. This applies only to certain authentication types. - -### Distance - -The numeric distance of the link, including a unit designation (e.g. 100 meters or 25 feet). diff --git a/docs/plugins/development/background-jobs.md b/docs/plugins/development/background-jobs.md new file mode 100644 index 000000000..810435268 --- /dev/null +++ b/docs/plugins/development/background-jobs.md @@ -0,0 +1,99 @@ +# Background Jobs + +!!! info "This feature was introduced in NetBox v4.1." + +NetBox plugins can defer certain operations by enqueuing [background jobs](../../features/background-jobs.md), which are executed asynchronously by background workers. This is helpful for decoupling long-running processes from the user-facing request-response cycle. + +For example, your plugin might need to fetch data from a remote system. Depending on the amount of data and the responsiveness of the remote server, this could take a few minutes. Deferring this task to a queued job ensures that it can be completed in the background, without interrupting the user. The data it fetches can be made available once the job has completed. + +## Job Runners + +A background job implements a basic [Job](../../models/core/job.md) executor for all kinds of tasks. It has logic implemented to handle the management of the associated job object, rescheduling of periodic jobs in the given interval and error handling. Adding custom jobs is done by subclassing NetBox's `JobRunner` class. + +::: utilities.jobs.JobRunner + +#### Example + +```python title="jobs.py" +from utilities.jobs import JobRunner + + +class MyTestJob(JobRunner): + class Meta: + name = "My Test Job" + + def run(self, *args, **kwargs): + obj = self.job.object + # your logic goes here +``` + +You can schedule the background job from within your code (e.g. from a model's `save()` method or a view) by calling `MyTestJob.enqueue()`. This method passes through all arguments to `Job.enqueue()`. However, no `name` argument must be passed, as the background job name will be used instead. + +### Attributes + +`JobRunner` attributes are defined under a class named `Meta` within the job. These are optional, but encouraged. + +#### `name` + +This is the human-friendly names of your background job. If omitted, the class name will be used. + +### Scheduled Jobs + +As described above, jobs can be scheduled for immediate execution or at any later time using the `enqueue()` method. However, for management purposes, the `enqueue_once()` method allows a job to be scheduled exactly once avoiding duplicates. If a job is already scheduled for a particular instance, a second one won't be scheduled, respecting thread safety. An example use case would be to schedule a periodic task that is bound to an instance in general, but not to any event of that instance (such as updates). The parameters of the `enqueue_once()` method are identical to those of `enqueue()`. + +!!! tip + It is not forbidden to `enqueue()` additional jobs while an interval schedule is active. An example use of this would be to schedule a periodic daily synchronization, but also trigger additional synchronizations on demand when the user presses a button. + +#### Example + +```python title="jobs.py" +from utilities.jobs import JobRunner + + +class MyHousekeepingJob(JobRunner): + class Meta: + name = "Housekeeping" + + def run(self, *args, **kwargs): + # your logic goes here +``` + +```python title="__init__.py" +from netbox.plugins import PluginConfig + +class MyPluginConfig(PluginConfig): + def ready(self): + from .jobs import MyHousekeepingJob + MyHousekeepingJob.setup(interval=60) +``` + +## Task queues + +Three task queues of differing priority are defined by default: + +* High +* Default +* Low + +Any tasks in the "high" queue are completed before the default queue is checked, and any tasks in the default queue are completed before those in the "low" queue. + +Plugins can also add custom queues for their own needs by setting the `queues` attribute under the PluginConfig class. An example is included below: + +```python +class MyPluginConfig(PluginConfig): + name = 'myplugin' + ... + queues = [ + 'foo', + 'bar', + ] +``` + +The `PluginConfig` above creates two custom queues with the following names `my_plugin.foo` and `my_plugin.bar`. (The plugin's name is prepended to each queue to avoid conflicts between plugins.) + +!!! warning "Configuring the RQ worker process" + By default, NetBox's RQ worker process only services the high, default, and low queues. Plugins which introduce custom queues should advise users to either reconfigure the default worker, or run a dedicated worker specifying the necessary queues. For example: + + ``` + python manage.py rqworker my_plugin.foo my_plugin.bar + ``` diff --git a/docs/plugins/development/background-tasks.md b/docs/plugins/development/background-tasks.md deleted file mode 100644 index 5ed05752a..000000000 --- a/docs/plugins/development/background-tasks.md +++ /dev/null @@ -1,30 +0,0 @@ -# Background Tasks - -NetBox supports the queuing of tasks that need to be performed in the background, decoupled from the request-response cycle, using the [Python RQ](https://python-rq.org/) library. Three task queues of differing priority are defined by default: - -* High -* Default -* Low - -Any tasks in the "high" queue are completed before the default queue is checked, and any tasks in the default queue are completed before those in the "low" queue. - -Plugins can also add custom queues for their own needs by setting the `queues` attribute under the PluginConfig class. An example is included below: - -```python -class MyPluginConfig(PluginConfig): - name = 'myplugin' - ... - queues = [ - 'foo', - 'bar', - ] -``` - -The PluginConfig above creates two custom queues with the following names `my_plugin.foo` and `my_plugin.bar`. (The plugin's name is prepended to each queue to avoid conflicts between plugins.) - -!!! warning "Configuring the RQ worker process" - By default, NetBox's RQ worker process only services the high, default, and low queues. Plugins which introduce custom queues should advise users to either reconfigure the default worker, or run a dedicated worker specifying the necessary queues. For example: - - ``` - python manage.py rqworker my_plugin.foo my_plugin.bar - ``` diff --git a/docs/plugins/development/events.md b/docs/plugins/development/event-types.md similarity index 52% rename from docs/plugins/development/events.md rename to docs/plugins/development/event-types.md index f41ed7ff5..4bcdeea31 100644 --- a/docs/plugins/development/events.md +++ b/docs/plugins/development/event-types.md @@ -1,16 +1,18 @@ -# Events +# Event Types -Plugins can register their own custom event types for use with NetBox [event rules](../../models/extras/eventrule.md). This is accomplished by calling the `register()` method on an instance of the `Event` class. This can be done anywhere within the plugin. An example is provided below. +!!! info "This feature was introduced in NetBox v4.1." + +Plugins can register their own custom event types for use with NetBox [event rules](../../models/extras/eventrule.md). This is accomplished by calling the `register()` method on an instance of the `EventType` class. This can be done anywhere within the plugin. An example is provided below. ```python from django.utils.translation import gettext_lazy as _ -from netbox.events import Event, EVENT_TYPE_SUCCESS +from netbox.events import EventType, EVENT_TYPE_KIND_SUCCESS -Event( +EventType( name='ticket_opened', text=_('Ticket opened'), - type=EVENT_TYPE_SUCCESS + kind=EVENT_TYPE_KIND_SUCCESS ).register() ``` -::: netbox.events.Event +::: netbox.events.EventType diff --git a/docs/plugins/development/index.md b/docs/plugins/development/index.md index c042be6ec..f3f9a3e4f 100644 --- a/docs/plugins/development/index.md +++ b/docs/plugins/development/index.md @@ -47,6 +47,7 @@ project-name/ - __init__.py - filtersets.py - graphql.py + - jobs.py - models.py - middleware.py - navigation.py diff --git a/docs/plugins/development/models.md b/docs/plugins/development/models.md index 902ee9c82..03cedda16 100644 --- a/docs/plugins/development/models.md +++ b/docs/plugins/development/models.md @@ -130,6 +130,8 @@ For more information about database migrations, see the [Django documentation](h ::: netbox.models.features.ExportTemplatesMixin +::: netbox.models.features.JobsMixin + ::: netbox.models.features.JournalingMixin ::: netbox.models.features.TagsMixin diff --git a/docs/plugins/development/views.md b/docs/plugins/development/views.md index cbf920ad5..1f5f164fd 100644 --- a/docs/plugins/development/views.md +++ b/docs/plugins/development/views.md @@ -203,7 +203,7 @@ Plugins can inject custom content into certain areas of core NetBox views. This | `right_page()` | Object view | Inject content on the right side of the page | | `full_width_page()` | Object view | Inject content across the entire bottom of the page | -!!! info "The `navbar()` method was introduced in NetBox v4.1." +!!! info "The `navbar()` and `alerts()` methods were introduced in NetBox v4.1." Additionally, a `render()` method is available for convenience. This method accepts the name of a template to render, and any additional context data you want to pass. Its use is optional, however. diff --git a/docs/release-notes/version-4.1.md b/docs/release-notes/version-4.1.md index 54bbf7ed9..dd846269d 100644 --- a/docs/release-notes/version-4.1.md +++ b/docs/release-notes/version-4.1.md @@ -5,17 +5,44 @@ ### Breaking Changes * Several filters deprecated in v4.0 have been removed (see [#15410](https://github.com/netbox-community/netbox/issues/15410)). -* The unit size for virtual disk size has been changed from 1 gigabyte to 1 megabyte. Existing values have been updated accordingly. +* The unit size for `VirtualMachine.disk` and `VirtualDisk.size` been changed from 1 gigabyte to 1 megabyte. Existing values have been updated accordingly. +* The `min_vid` and `max_vid` fields on the VLAN group model have been replaced with `vid_ranges`, an array of starting and ending integer pairs. +* The five individual event type fields on the EventRule model have been replaced by a single `event_types` array field, indicating each assigned event type by name. +* The `validate()` method on CustomValidator subclasses now **must** accept the request argument (deprecated in v4.0 by #14279). ### New Features +#### Circuit Groups ([#7025](https://github.com/netbox-community/netbox/issues/7025)) + +Circuits can now be assigned to groups for administrative purposes. Each circuit may be assigned to multiple groups, and each assignment may optionally indicate a priority (primary, secondary, or tertiary). + +#### VLAN Group ID Ranges ([#9627](https://github.com/netbox-community/netbox/issues/9627)) + +The VLAN group model has been enhanced to support multiple VLAN ID (VID) ranges, whereas previously it could track only a single beginning and ending VID. VID ranges are stored as an array of beginning and ending (inclusive) integers. + +#### Rack Types ([#12826](https://github.com/netbox-community/netbox/issues/12826)) + +A new rack type model has been introduced, which functions similar to the device type model. Users can now define a common make and model of rack, the attributes of which are automatically populated when creating a new rack of that type. + +#### Plugins Catalog Integration ([#14731](https://github.com/netbox-community/netbox/issues/14731)) + +The NetBox UI now integrates directly with the canonical plugins catalog hosted by NetBox Labs. In addition to locally installed plugins, users can explore available plugins and check for newer releases. + +#### User Notifications ([#15621](https://github.com/netbox-community/netbox/issues/15621)) + +NetBox now includes a user notification system. Users can subscribe to individual objects and be alerted to changes live within the web interface. Additionally, event rules can now trigger notifications to specific users and/or groups. Plugins can also employ this notification system for their own purposes. + ### Enhancements * [#7537](https://github.com/netbox-community/netbox/issues/7537) - Add a serial number field for virtual machines * [#8984](https://github.com/netbox-community/netbox/issues/8984) - Enable filtering of custom script output by log level +* [#11969](https://github.com/netbox-community/netbox/issues/11969) - Support for tracking airflow on racks and module types * [#15156](https://github.com/netbox-community/netbox/issues/15156) - Add `display_url` field to all REST API serializers * [#16359](https://github.com/netbox-community/netbox/issues/16359) - Enable plugins to embed content in the top navigation bar * [#16580](https://github.com/netbox-community/netbox/issues/16580) - Enable individual views to enforce `LOGIN_REQUIRED` selectively (remove `AUTH_EXEMPT_PATHS`) +* [#16776](https://github.com/netbox-community/netbox/issues/16776) - Added an `alerts()` method to `PluginTemplateExtension` for embedding important information about specific objects +* [#16782](https://github.com/netbox-community/netbox/issues/16782) - Enable filtering of selection choices for object type custom fields +* [#16866](https://github.com/netbox-community/netbox/issues/16866) - Introduced a mechanism for plugins to register custom event types (for use with user notifications) ### Plugins @@ -24,13 +51,34 @@ ### Other Changes * [#14692](https://github.com/netbox-community/netbox/issues/14692) - Change atomic unit for virtual disks from 1GB to 1MB +* [#14861](https://github.com/netbox-community/netbox/issues/14861) - The URL path for UI views concerning virtual disks has been standardized to `/virtualization/virtual-disks/` * [#15410](https://github.com/netbox-community/netbox/issues/15410) - Removed various deprecated filters * [#15908](https://github.com/netbox-community/netbox/issues/15908) - Indicate product edition in release data * [#16388](https://github.com/netbox-community/netbox/issues/16388) - Move all change logging resources from `extras` to `core` +* [#16884](https://github.com/netbox-community/netbox/issues/16884) - Remove the ID column from the default table configuration for changelog records ### REST API Changes * The `/api/extras/object-changes/` endpoint has moved to `/api/core/object-changes/` +* Added the following endpoints: + * `/api/circuits/circuit-groups/` + * `/api/circuits/circuit-group-assignments/` + * `/api/dcim/rack-types/` +* circuits.Circuit + * Added the `assignments` field, which lists all group assignments +* dcim.ModuleType + * Added the optional `airflow` choice field +* dcim.Rack + * Added the optional `rack_type` foreign key field + * Added the optional `airflow` choice field +* extras.CustomField + * Added the `related_object_filter` JSON field for object and multi-object custom fields +* extras.EventRule + * Removed the `type_create`, `type_update`, `type_delete`, `type_job_start`, and `type_job_end` boolean fields + * Added the `event_types` array field +* ipam.VLANGroup + * Removed the `min_vid` and `max_vid` fields + * Added the `vid_ranges` field, and array of starting & ending VLAN IDs * virtualization.VirtualMachine * Added the optional `serial` field * wireless.WirelessLink diff --git a/mkdocs.yml b/mkdocs.yml index 841a9df47..072c564e8 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -86,6 +86,7 @@ nav: - Change Logging: 'features/change-logging.md' - Journaling: 'features/journaling.md' - Event Rules: 'features/event-rules.md' + - Notifications: 'features/notifications.md' - Background Jobs: 'features/background-jobs.md' - Auth & Permissions: 'features/authentication-permissions.md' - API & Integration: 'features/api-integration.md' @@ -142,11 +143,11 @@ nav: - Forms: 'plugins/development/forms.md' - Filters & Filter Sets: 'plugins/development/filtersets.md' - Search: 'plugins/development/search.md' - - Events: 'plugins/development/events.md' + - Event Types: 'plugins/development/event-types.md' - Data Backends: 'plugins/development/data-backends.md' - REST API: 'plugins/development/rest-api.md' - GraphQL API: 'plugins/development/graphql-api.md' - - Background Tasks: 'plugins/development/background-tasks.md' + - Background Jobs: 'plugins/development/background-jobs.md' - Dashboard Widgets: 'plugins/development/dashboard-widgets.md' - Staged Changes: 'plugins/development/staged-changes.md' - Exceptions: 'plugins/development/exceptions.md' diff --git a/netbox/circuits/forms/model_forms.py b/netbox/circuits/forms/model_forms.py index 554f2af5a..9a54fdccb 100644 --- a/netbox/circuits/forms/model_forms.py +++ b/netbox/circuits/forms/model_forms.py @@ -198,6 +198,7 @@ class CircuitGroupAssignmentForm(NetBoxModelForm): circuit = DynamicModelChoiceField( label=_('Circuit'), queryset=Circuit.objects.all(), + selector=True ) class Meta: diff --git a/netbox/circuits/migrations/0044_circuitgroup_circuitgroupassignment_and_more.py b/netbox/circuits/migrations/0044_circuit_groups.py similarity index 98% rename from netbox/circuits/migrations/0044_circuitgroup_circuitgroupassignment_and_more.py rename to netbox/circuits/migrations/0044_circuit_groups.py index 40ea5bd1e..f3984877f 100644 --- a/netbox/circuits/migrations/0044_circuitgroup_circuitgroupassignment_and_more.py +++ b/netbox/circuits/migrations/0044_circuit_groups.py @@ -78,7 +78,7 @@ class Migration(migrations.Migration): options={ 'verbose_name': 'Circuit group assignment', 'verbose_name_plural': 'Circuit group assignments', - 'ordering': ('circuit', 'priority', 'pk'), + 'ordering': ('group', 'circuit', 'priority', 'pk'), }, ), migrations.AddConstraint( diff --git a/netbox/circuits/models/circuits.py b/netbox/circuits/models/circuits.py index 7c5e5f2b5..68c938aa9 100644 --- a/netbox/circuits/models/circuits.py +++ b/netbox/circuits/models/circuits.py @@ -203,7 +203,7 @@ class CircuitGroupAssignment(CustomFieldsMixin, ExportTemplatesMixin, TagsMixin, ) class Meta: - ordering = ('circuit', 'priority', 'pk') + ordering = ('group', 'circuit', 'priority', 'pk') constraints = ( models.UniqueConstraint( fields=('circuit', 'group'), diff --git a/netbox/circuits/tables/circuits.py b/netbox/circuits/tables/circuits.py index 3145df43e..1c9791283 100644 --- a/netbox/circuits/tables/circuits.py +++ b/netbox/circuits/tables/circuits.py @@ -77,18 +77,22 @@ class CircuitTable(TenancyColumnsMixin, ContactsColumnMixin, NetBoxTable): verbose_name=_('Commit Rate') ) comments = columns.MarkdownColumn( - verbose_name=_('Comments'), + verbose_name=_('Comments') ) tags = columns.TagColumn( url_name='circuits:circuit_list' ) + assignments = columns.ManyToManyColumn( + verbose_name=_('Assignments'), + linkify_item=True + ) class Meta(NetBoxTable.Meta): model = Circuit fields = ( 'pk', 'id', 'cid', 'provider', 'provider_account', 'type', 'status', 'tenant', 'tenant_group', 'termination_a', 'termination_z', 'install_date', 'termination_date', 'commit_rate', 'description', - 'comments', 'contacts', 'tags', 'created', 'last_updated', + 'comments', 'contacts', 'tags', 'created', 'last_updated', 'assignments', ) default_columns = ( 'pk', 'cid', 'provider', 'type', 'status', 'tenant', 'termination_a', 'termination_z', 'description', diff --git a/netbox/core/api/views.py b/netbox/core/api/views.py index ff488e3cd..b3a024c02 100644 --- a/netbox/core/api/views.py +++ b/netbox/core/api/views.py @@ -7,6 +7,8 @@ from rest_framework.routers import APIRootView from rest_framework.viewsets import ReadOnlyModelViewSet from core import filtersets +from core.choices import DataSourceStatusChoices +from core.jobs import SyncDataSourceJob from core.models import * from netbox.api.metadata import ContentTypeMetadata from netbox.api.viewsets import NetBoxModelViewSet, NetBoxReadOnlyModelViewSet @@ -36,7 +38,11 @@ class DataSourceViewSet(NetBoxModelViewSet): if not request.user.has_perm('core.sync_datasource', obj=datasource): raise PermissionDenied(_("This user does not have permission to synchronize this data source.")) - datasource.enqueue_sync_job(request) + # Enqueue the sync job & update the DataSource's status + SyncDataSourceJob.enqueue(instance=datasource, user=request.user) + datasource.status = DataSourceStatusChoices.QUEUED + DataSource.objects.filter(pk=datasource.pk).update(status=datasource.status) + serializer = serializers.DataSourceSerializer(datasource, context={'request': request}) return Response(serializer.data) diff --git a/netbox/core/choices.py b/netbox/core/choices.py index ee0febaff..01a072ce1 100644 --- a/netbox/core/choices.py +++ b/netbox/core/choices.py @@ -59,6 +59,12 @@ class JobStatusChoices(ChoiceSet): (STATUS_FAILED, _('Failed'), 'red'), ) + ENQUEUED_STATE_CHOICES = ( + STATUS_PENDING, + STATUS_SCHEDULED, + STATUS_RUNNING, + ) + TERMINAL_STATE_CHOICES = ( STATUS_COMPLETED, STATUS_ERRORED, diff --git a/netbox/core/events.py b/netbox/core/events.py index 4855e5b23..98a1ea18f 100644 --- a/netbox/core/events.py +++ b/netbox/core/events.py @@ -1,6 +1,6 @@ from django.utils.translation import gettext as _ -from netbox.events import Event, EVENT_TYPE_DANGER, EVENT_TYPE_SUCCESS, EVENT_TYPE_WARNING +from netbox.events import EventType, EVENT_TYPE_KIND_DANGER, EVENT_TYPE_KIND_SUCCESS, EVENT_TYPE_KIND_WARNING __all__ = ( 'JOB_COMPLETED', @@ -24,10 +24,10 @@ JOB_FAILED = 'job_failed' JOB_ERRORED = 'job_errored' # Register core events -Event(OBJECT_CREATED, _('Object created')).register() -Event(OBJECT_UPDATED, _('Object updated')).register() -Event(OBJECT_DELETED, _('Object deleted')).register() -Event(JOB_STARTED, _('Job started')).register() -Event(JOB_COMPLETED, _('Job completed'), type=EVENT_TYPE_SUCCESS).register() -Event(JOB_FAILED, _('Job failed'), type=EVENT_TYPE_WARNING).register() -Event(JOB_ERRORED, _('Job errored'), type=EVENT_TYPE_DANGER).register() +EventType(OBJECT_CREATED, _('Object created')).register() +EventType(OBJECT_UPDATED, _('Object updated')).register() +EventType(OBJECT_DELETED, _('Object deleted')).register() +EventType(JOB_STARTED, _('Job started')).register() +EventType(JOB_COMPLETED, _('Job completed'), kind=EVENT_TYPE_KIND_SUCCESS).register() +EventType(JOB_FAILED, _('Job failed'), kind=EVENT_TYPE_KIND_WARNING).register() +EventType(JOB_ERRORED, _('Job errored'), kind=EVENT_TYPE_KIND_DANGER).register() diff --git a/netbox/core/jobs.py b/netbox/core/jobs.py index 264313e62..1c38cf61f 100644 --- a/netbox/core/jobs.py +++ b/netbox/core/jobs.py @@ -1,33 +1,33 @@ import logging from netbox.search.backends import search_backend -from .choices import * +from utilities.jobs import JobRunner +from .choices import DataSourceStatusChoices from .exceptions import SyncError from .models import DataSource -from rq.timeouts import JobTimeoutException logger = logging.getLogger(__name__) -def sync_datasource(job, *args, **kwargs): +class SyncDataSourceJob(JobRunner): """ Call sync() on a DataSource. """ - datasource = DataSource.objects.get(pk=job.object_id) - try: - job.start() - datasource.sync() + class Meta: + name = 'Synchronization' - # Update the search cache for DataFiles belonging to this source - search_backend.cache(datasource.datafiles.iterator()) + def run(self, *args, **kwargs): + datasource = DataSource.objects.get(pk=self.job.object_id) - job.terminate() + try: + datasource.sync() - except Exception as e: - job.terminate(status=JobStatusChoices.STATUS_ERRORED, error=repr(e)) - DataSource.objects.filter(pk=datasource.pk).update(status=DataSourceStatusChoices.FAILED) - if type(e) in (SyncError, JobTimeoutException): - logging.error(e) - else: + # Update the search cache for DataFiles belonging to this source + search_backend.cache(datasource.datafiles.iterator()) + + except Exception as e: + DataSource.objects.filter(pk=datasource.pk).update(status=DataSourceStatusChoices.FAILED) + if type(e) is SyncError: + logging.error(e) raise e diff --git a/netbox/core/migrations/0012_job_object_type_optional.py b/netbox/core/migrations/0012_job_object_type_optional.py new file mode 100644 index 000000000..3c6664afc --- /dev/null +++ b/netbox/core/migrations/0012_job_object_type_optional.py @@ -0,0 +1,24 @@ +import django.db.models.deletion +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('contenttypes', '0002_remove_content_type_name'), + ('core', '0011_move_objectchange'), + ] + + operations = [ + migrations.AlterField( + model_name='job', + name='object_type', + field=models.ForeignKey( + blank=True, + null=True, + on_delete=django.db.models.deletion.CASCADE, + related_name='jobs', + to='contenttypes.contenttype' + ), + ), + ] diff --git a/netbox/core/models/data.py b/netbox/core/models/data.py index 48fa2ff71..a8e90ec3f 100644 --- a/netbox/core/models/data.py +++ b/netbox/core/models/data.py @@ -1,10 +1,10 @@ import hashlib import logging import os -import yaml from fnmatch import fnmatchcase from urllib.parse import urlparse +import yaml from django.conf import settings from django.contrib.contenttypes.fields import GenericForeignKey from django.core.exceptions import ValidationError @@ -12,7 +12,6 @@ from django.core.validators import RegexValidator from django.db import models from django.urls import reverse from django.utils import timezone -from django.utils.module_loading import import_string from django.utils.translation import gettext as _ from netbox.constants import CENSOR_TOKEN, CENSOR_TOKEN_CHANGED @@ -23,7 +22,6 @@ from utilities.querysets import RestrictedQuerySet from ..choices import * from ..exceptions import SyncError from ..signals import post_sync, pre_sync -from .jobs import Job __all__ = ( 'AutoSyncRecord', @@ -153,21 +151,6 @@ class DataSource(JobsMixin, PrimaryModel): return objectchange - def enqueue_sync_job(self, request): - """ - Enqueue a background job to synchronize the DataSource by calling sync(). - """ - # Set the status to "syncing" - self.status = DataSourceStatusChoices.QUEUED - DataSource.objects.filter(pk=self.pk).update(status=self.status) - - # Enqueue a sync job - return Job.enqueue( - import_string('core.jobs.sync_datasource'), - instance=self, - user=request.user - ) - def get_backend(self): backend_params = self.parameters or {} return self.backend_class(self.source_url, **backend_params) diff --git a/netbox/core/models/jobs.py b/netbox/core/models/jobs.py index c5fbb918c..1d0e7fdeb 100644 --- a/netbox/core/models/jobs.py +++ b/netbox/core/models/jobs.py @@ -31,6 +31,8 @@ class Job(models.Model): to='contenttypes.ContentType', related_name='jobs', on_delete=models.CASCADE, + blank=True, + null=True ) object_id = models.PositiveBigIntegerField( blank=True, @@ -197,25 +199,34 @@ class Job(models.Model): job_end.send(self) @classmethod - def enqueue(cls, func, instance, name='', user=None, schedule_at=None, interval=None, **kwargs): + def enqueue(cls, func, instance=None, name='', user=None, schedule_at=None, interval=None, immediate=False, **kwargs): """ Create a Job instance and enqueue a job using the given callable Args: func: The callable object to be enqueued for execution - instance: The NetBox object to which this job pertains + instance: The NetBox object to which this job pertains (optional) name: Name for the job (optional) user: The user responsible for running the job schedule_at: Schedule the job to be executed at the passed date and time interval: Recurrence interval (in minutes) + immediate: Run the job immediately without scheduling it in the background. Should be used for interactive + management commands only. """ - object_type = ObjectType.objects.get_for_model(instance, for_concrete_model=False) - rq_queue_name = get_queue_for_model(object_type.model) + if schedule_at and immediate: + raise ValueError("enqueue() cannot be called with values for both schedule_at and immediate.") + + if instance: + object_type = ObjectType.objects.get_for_model(instance, for_concrete_model=False) + object_id = instance.pk + else: + object_type = object_id = None + rq_queue_name = get_queue_for_model(object_type.model if object_type else None) queue = django_rq.get_queue(rq_queue_name) status = JobStatusChoices.STATUS_SCHEDULED if schedule_at else JobStatusChoices.STATUS_PENDING job = Job.objects.create( object_type=object_type, - object_id=instance.pk, + object_id=object_id, name=name, status=status, scheduled=schedule_at, @@ -224,8 +235,16 @@ class Job(models.Model): job_id=uuid.uuid4() ) - if schedule_at: + # Run the job immediately, rather than enqueuing it as a background task. Note that this is a synchronous + # (blocking) operation, and execution will pause until the job completes. + if immediate: + func(job_id=str(job.job_id), job=job, **kwargs) + + # Schedule the job to run at a specific date & time. + elif schedule_at: queue.enqueue_at(schedule_at, func, job_id=str(job.job_id), job=job, **kwargs) + + # Schedule the job to run asynchronously at this first available opportunity. else: queue.enqueue(func, job_id=str(job.job_id), job=job, **kwargs) diff --git a/netbox/core/plugins.py b/netbox/core/plugins.py index ab8fd97a1..374cfbe9a 100644 --- a/netbox/core/plugins.py +++ b/netbox/core/plugins.py @@ -155,7 +155,6 @@ def get_catalog_plugins(): # Populate author (if any) if data['author']: - print(data['author']) author = PluginAuthor( name=data['author']['name'], org_id=data['author']['org_id'], diff --git a/netbox/core/tables/plugins.py b/netbox/core/tables/plugins.py index 529fe60f4..f89a886d2 100644 --- a/netbox/core/tables/plugins.py +++ b/netbox/core/tables/plugins.py @@ -44,7 +44,7 @@ class CatalogPluginTable(BaseTable): verbose_name=_('Name') ) author = tables.Column( - accessor=tables.A('author.name'), + accessor=tables.A('author__name'), verbose_name=_('Author') ) is_local = columns.BooleanColumn( diff --git a/netbox/core/views.py b/netbox/core/views.py index 79ef33e1d..06c8b00d0 100644 --- a/netbox/core/views.py +++ b/netbox/core/views.py @@ -34,6 +34,8 @@ from utilities.htmx import htmx_partial from utilities.query import count_related from utilities.views import ContentTypePermissionRequiredMixin, GetRelatedModelsMixin, register_model_view from . import filtersets, forms, tables +from .choices import DataSourceStatusChoices +from .jobs import SyncDataSourceJob from .models import * from .plugins import get_plugins from .tables import CatalogPluginTable, PluginVersionTable @@ -76,7 +78,11 @@ class DataSourceSyncView(BaseObjectView): def post(self, request, pk): datasource = get_object_or_404(self.queryset, pk=pk) - job = datasource.enqueue_sync_job(request) + + # Enqueue the sync job & update the DataSource's status + job = SyncDataSourceJob.enqueue(instance=datasource, user=request.user) + datasource.status = DataSourceStatusChoices.QUEUED + DataSource.objects.filter(pk=datasource.pk).update(status=datasource.status) messages.success(request, f"Queued job #{job.pk} to sync {datasource}") return redirect(datasource.get_absolute_url()) diff --git a/netbox/dcim/filtersets.py b/netbox/dcim/filtersets.py index bcd1f23b2..5e404179a 100644 --- a/netbox/dcim/filtersets.py +++ b/netbox/dcim/filtersets.py @@ -375,6 +375,17 @@ class RackFilterSet(NetBoxModelFilterSet, TenancyFilterSet, ContactModelFilterSe to_field_name='slug', label=_('Location (slug)'), ) + manufacturer_id = django_filters.ModelMultipleChoiceFilter( + field_name='rack_type__manufacturer', + queryset=Manufacturer.objects.all(), + label=_('Manufacturer (ID)'), + ) + manufacturer = django_filters.ModelMultipleChoiceFilter( + field_name='rack_type__manufacturer__slug', + queryset=Manufacturer.objects.all(), + to_field_name='slug', + label=_('Manufacturer (slug)'), + ) rack_type = django_filters.ModelMultipleChoiceFilter( field_name='rack_type__slug', queryset=RackType.objects.all(), diff --git a/netbox/dcim/forms/filtersets.py b/netbox/dcim/forms/filtersets.py index c8124aa11..b43ed85ce 100644 --- a/netbox/dcim/forms/filtersets.py +++ b/netbox/dcim/forms/filtersets.py @@ -312,8 +312,8 @@ class RackFilterForm(TenancyFilterForm, ContactModelFilterForm, RackBaseFilterFo FieldSet('q', 'filter_id', 'tag'), FieldSet('region_id', 'site_group_id', 'site_id', 'location_id', name=_('Location')), FieldSet('tenant_group_id', 'tenant_id', name=_('Tenant')), - FieldSet('status', 'role_id', 'serial', 'asset_tag', name=_('Rack')), - FieldSet('form_factor', 'width', 'u_height', 'airflow', name=_('Rack Type')), + FieldSet('status', 'role_id', 'manufacturer_id', 'rack_type_id', 'serial', 'asset_tag', name=_('Rack')), + FieldSet('form_factor', 'width', 'u_height', 'airflow', name=_('Hardware')), FieldSet('starting_unit', 'desc_units', name=_('Numbering')), FieldSet('weight', 'max_weight', 'weight_unit', name=_('Weight')), FieldSet('contact', 'contact_role', 'contact_group', name=_('Contacts')), @@ -357,6 +357,19 @@ class RackFilterForm(TenancyFilterForm, ContactModelFilterForm, RackBaseFilterFo null_option='None', label=_('Role') ) + manufacturer_id = DynamicModelMultipleChoiceField( + queryset=Manufacturer.objects.all(), + required=False, + label=_('Manufacturer') + ) + rack_type_id = DynamicModelMultipleChoiceField( + queryset=RackType.objects.all(), + required=False, + query_params={ + 'manufacturer_id': '$manufacturer_id' + }, + label=_('Rack type') + ) serial = forms.CharField( label=_('Serial'), required=False diff --git a/netbox/dcim/models/devices.py b/netbox/dcim/models/devices.py index a790cceef..c6e97a3cd 100644 --- a/netbox/dcim/models/devices.py +++ b/netbox/dcim/models/devices.py @@ -417,6 +417,10 @@ class ModuleType(ImageAttachmentsMixin, PrimaryModel, WeightMixin): def get_absolute_url(self): return reverse('dcim:moduletype', args=[self.pk]) + @property + def full_name(self): + return f"{self.manufacturer} {self.model}" + def to_yaml(self): data = { 'manufacturer': self.manufacturer.name, diff --git a/netbox/dcim/models/racks.py b/netbox/dcim/models/racks.py index cfa8f28be..8457271f4 100644 --- a/netbox/dcim/models/racks.py +++ b/netbox/dcim/models/racks.py @@ -152,8 +152,8 @@ class RackType(RackBase): ) clone_fields = ( - 'manufacturer', 'form_factor', 'width', 'u_height', 'desc_units', 'outer_width', 'outer_depth', 'outer_unit', - 'mounting_depth', 'weight', 'max_weight', 'weight_unit', + 'manufacturer', 'form_factor', 'width', 'u_height', 'airflow', 'desc_units', 'outer_width', 'outer_depth', + 'outer_unit', 'mounting_depth', 'weight', 'max_weight', 'weight_unit', ) prerequisite_models = ( 'dcim.Manufacturer', @@ -170,6 +170,10 @@ class RackType(RackBase): def get_absolute_url(self): return reverse('dcim:racktype', args=[self.pk]) + @property + def full_name(self): + return f"{self.manufacturer} {self.name}" + def clean(self): super().clean() diff --git a/netbox/dcim/tables/racks.py b/netbox/dcim/tables/racks.py index 064a5a43d..b8295c286 100644 --- a/netbox/dcim/tables/racks.py +++ b/netbox/dcim/tables/racks.py @@ -84,6 +84,11 @@ class RackTypeTable(NetBoxTable): comments = columns.MarkdownColumn( verbose_name=_('Comments'), ) + instance_count = columns.LinkedCountColumn( + viewname='dcim:rack_list', + url_params={'rack_type_id': 'pk'}, + verbose_name=_('Instances') + ) tags = columns.TagColumn( url_name='dcim:rack_list' ) @@ -92,11 +97,11 @@ class RackTypeTable(NetBoxTable): model = RackType fields = ( 'pk', 'id', 'name', 'manufacturer', 'form_factor', 'u_height', 'starting_unit', 'width', 'outer_width', - 'outer_depth', 'mounting_depth', 'airflow', 'weight', 'max_weight', 'description', 'comments', 'tags', - 'created', 'last_updated', + 'outer_depth', 'mounting_depth', 'airflow', 'weight', 'max_weight', 'description', 'comments', + 'instance_count', 'tags', 'created', 'last_updated', ) default_columns = ( - 'pk', 'name', 'manufacturer', 'type', 'u_height', 'description', + 'pk', 'name', 'manufacturer', 'type', 'u_height', 'description', 'instance_count', ) @@ -124,6 +129,15 @@ class RackTable(TenancyColumnsMixin, ContactsColumnMixin, NetBoxTable): role = columns.ColoredLabelColumn( verbose_name=_('Role'), ) + manufacturer = tables.Column( + verbose_name=_('Manufacturer'), + accessor=Accessor('rack_type__manufacturer'), + linkify=True + ) + rack_type = tables.Column( + linkify=True, + verbose_name=_('Type') + ) u_height = tables.TemplateColumn( template_code="{{ value }}U", verbose_name=_('Height') @@ -169,14 +183,14 @@ class RackTable(TenancyColumnsMixin, ContactsColumnMixin, NetBoxTable): class Meta(NetBoxTable.Meta): model = Rack fields = ( - 'pk', 'id', 'name', 'site', 'location', 'status', 'facility_id', 'tenant', 'tenant_group', 'role', 'serial', - 'asset_tag', 'form_factor', 'u_height', 'starting_unit', 'width', 'outer_width', 'outer_depth', - 'mounting_depth', 'airflow', 'weight', 'max_weight', 'comments', 'device_count', 'get_utilization', - 'get_power_utilization', 'description', 'contacts', 'tags', 'created', 'last_updated', + 'pk', 'id', 'name', 'site', 'location', 'status', 'facility_id', 'tenant', 'tenant_group', 'role', + 'rack_type', 'serial', 'asset_tag', 'form_factor', 'u_height', 'starting_unit', 'width', 'outer_width', + 'outer_depth', 'mounting_depth', 'airflow', 'weight', 'max_weight', 'comments', 'device_count', + 'get_utilization', 'get_power_utilization', 'description', 'contacts', 'tags', 'created', 'last_updated', ) default_columns = ( - 'pk', 'name', 'site', 'location', 'status', 'facility_id', 'tenant', 'role', 'u_height', 'device_count', - 'get_utilization', + 'pk', 'name', 'site', 'location', 'status', 'facility_id', 'tenant', 'role', 'rack_type', 'u_height', + 'device_count', 'get_utilization', ) diff --git a/netbox/dcim/views.py b/netbox/dcim/views.py index 5ff20f35e..5ad619452 100644 --- a/netbox/dcim/views.py +++ b/netbox/dcim/views.py @@ -584,7 +584,9 @@ class RackRoleBulkDeleteView(generic.BulkDeleteView): # class RackTypeListView(generic.ObjectListView): - queryset = RackType.objects.all() + queryset = RackType.objects.annotate( + instance_count=count_related(Rack, 'rack_type') + ) filterset = filtersets.RackTypeFilterSet filterset_form = forms.RackTypeFilterForm table = tables.RackTypeTable diff --git a/netbox/extras/api/serializers_/customfields.py b/netbox/extras/api/serializers_/customfields.py index 9675cb173..2c8e7c127 100644 --- a/netbox/extras/api/serializers_/customfields.py +++ b/netbox/extras/api/serializers_/customfields.py @@ -62,7 +62,7 @@ class CustomFieldSerializer(ValidatedModelSerializer): fields = [ 'id', 'url', 'display_url', 'display', 'object_types', 'type', 'related_object_type', 'data_type', 'name', 'label', 'group_name', 'description', 'required', 'search_weight', 'filter_logic', 'ui_visible', - 'ui_editable', 'is_cloneable', 'default', 'weight', 'validation_minimum', 'validation_maximum', + 'ui_editable', 'is_cloneable', 'default', 'related_object_filter', 'weight', 'validation_minimum', 'validation_maximum', 'validation_regex', 'validation_unique', 'choice_set', 'comments', 'created', 'last_updated', ] brief_fields = ('id', 'url', 'display', 'name', 'description') diff --git a/netbox/extras/api/views.py b/netbox/extras/api/views.py index 2369e8f10..142be1b8a 100644 --- a/netbox/extras/api/views.py +++ b/netbox/extras/api/views.py @@ -1,5 +1,6 @@ from django.http import Http404 from django.shortcuts import get_object_or_404 +from django.utils.module_loading import import_string from django_rq.queues import get_connection from rest_framework import status from rest_framework.decorators import action @@ -11,10 +12,10 @@ from rest_framework.routers import APIRootView from rest_framework.viewsets import ModelViewSet, ReadOnlyModelViewSet from rq import Worker -from core.models import Job, ObjectType +from core.models import ObjectType from extras import filtersets from extras.models import * -from extras.scripts import run_script +from extras.jobs import ScriptJob from netbox.api.authentication import IsAuthenticatedOrLoginNotRequired from netbox.api.features import SyncedDataMixin from netbox.api.metadata import ContentTypeMetadata @@ -273,10 +274,8 @@ class ScriptViewSet(ModelViewSet): raise RQWorkerNotRunningException() if input_serializer.is_valid(): - Job.enqueue( - run_script, + ScriptJob.enqueue( instance=script, - name=script.python_class.class_name, user=request.user, data=input_serializer.data['data'], request=copy_safe_request(request), diff --git a/netbox/extras/choices.py b/netbox/extras/choices.py index 387716c85..4525d8689 100644 --- a/netbox/extras/choices.py +++ b/netbox/extras/choices.py @@ -156,16 +156,16 @@ class LogLevelChoices(ChoiceSet): LOG_DEBUG = 'debug' LOG_DEFAULT = 'default' - LOG_SUCCESS = 'success' LOG_INFO = 'info' + LOG_SUCCESS = 'success' LOG_WARNING = 'warning' LOG_FAILURE = 'failure' CHOICES = ( (LOG_DEBUG, _('Debug'), 'teal'), (LOG_DEFAULT, _('Default'), 'gray'), - (LOG_SUCCESS, _('Success'), 'green'), (LOG_INFO, _('Info'), 'cyan'), + (LOG_SUCCESS, _('Success'), 'green'), (LOG_WARNING, _('Warning'), 'yellow'), (LOG_FAILURE, _('Failure'), 'red'), ) @@ -173,8 +173,8 @@ class LogLevelChoices(ChoiceSet): SYSTEM_LEVELS = { LOG_DEBUG: logging.DEBUG, LOG_DEFAULT: logging.INFO, - LOG_SUCCESS: logging.INFO, LOG_INFO: logging.INFO, + LOG_SUCCESS: logging.INFO, LOG_WARNING: logging.WARNING, LOG_FAILURE: logging.ERROR, } @@ -191,35 +191,6 @@ class DurationChoices(ChoiceSet): ) -# -# Job results -# - -class JobResultStatusChoices(ChoiceSet): - - STATUS_PENDING = 'pending' - STATUS_SCHEDULED = 'scheduled' - STATUS_RUNNING = 'running' - STATUS_COMPLETED = 'completed' - STATUS_ERRORED = 'errored' - STATUS_FAILED = 'failed' - - CHOICES = ( - (STATUS_PENDING, _('Pending'), 'cyan'), - (STATUS_SCHEDULED, _('Scheduled'), 'gray'), - (STATUS_RUNNING, _('Running'), 'blue'), - (STATUS_COMPLETED, _('Completed'), 'green'), - (STATUS_ERRORED, _('Errored'), 'red'), - (STATUS_FAILED, _('Failed'), 'red'), - ) - - TERMINAL_STATE_CHOICES = ( - STATUS_COMPLETED, - STATUS_ERRORED, - STATUS_FAILED, - ) - - # # Webhooks # diff --git a/netbox/extras/constants.py b/netbox/extras/constants.py index e8e2c6d8a..3bfe3b21b 100644 --- a/netbox/extras/constants.py +++ b/netbox/extras/constants.py @@ -136,10 +136,10 @@ DEFAULT_DASHBOARD = [ ] LOG_LEVEL_RANK = { - LogLevelChoices.LOG_DEFAULT: 0, - LogLevelChoices.LOG_DEBUG: 1, - LogLevelChoices.LOG_SUCCESS: 2, - LogLevelChoices.LOG_INFO: 3, + LogLevelChoices.LOG_DEBUG: 0, + LogLevelChoices.LOG_DEFAULT: 1, + LogLevelChoices.LOG_INFO: 2, + LogLevelChoices.LOG_SUCCESS: 3, LogLevelChoices.LOG_WARNING: 4, LogLevelChoices.LOG_FAILURE: 5, } diff --git a/netbox/extras/events.py b/netbox/extras/events.py index de6a88c1b..e80f4a558 100644 --- a/netbox/extras/events.py +++ b/netbox/extras/events.py @@ -1,5 +1,6 @@ from collections import defaultdict import logging +from collections import defaultdict from django.conf import settings from django.contrib.auth import get_user_model @@ -10,7 +11,6 @@ from django.utils.translation import gettext as _ from django_rq import get_queue from core.events import * -from core.models import Job from netbox.config import get_config from netbox.constants import RQ_QUEUE_DEFAULT from netbox.registry import registry @@ -126,8 +126,8 @@ def process_event_rules(event_rules, object_type, event_type, data, username=Non script = event_rule.action_object.python_class() # Enqueue a Job to record the script's execution - Job.enqueue( - "extras.scripts.run_script", + from extras.jobs import ScriptJob + ScriptJob.enqueue( instance=event_rule.action_object, name=script.name, user=user, diff --git a/netbox/extras/forms/model_forms.py b/netbox/extras/forms/model_forms.py index b4221b960..ce013f7c6 100644 --- a/netbox/extras/forms/model_forms.py +++ b/netbox/extras/forms/model_forms.py @@ -67,7 +67,7 @@ class CustomFieldForm(forms.ModelForm): FieldSet( 'search_weight', 'filter_logic', 'ui_visible', 'ui_editable', 'weight', 'is_cloneable', name=_('Behavior') ), - FieldSet('default', 'choice_set', name=_('Values')), + FieldSet('default', 'choice_set', 'related_object_filter', name=_('Values')), FieldSet( 'validation_minimum', 'validation_maximum', 'validation_regex', 'validation_unique', name=_('Validation') ), diff --git a/netbox/extras/jobs.py b/netbox/extras/jobs.py new file mode 100644 index 000000000..62f8f6959 --- /dev/null +++ b/netbox/extras/jobs.py @@ -0,0 +1,107 @@ +import logging +import traceback +from contextlib import nullcontext + +from django.db import transaction +from django.utils.translation import gettext as _ + +from extras.models import Script as ScriptModel +from extras.signals import clear_events +from netbox.context_managers import event_tracking +from utilities.exceptions import AbortScript, AbortTransaction +from utilities.jobs import JobRunner +from .utils import is_report + + +class ScriptJob(JobRunner): + """ + Script execution job. + + A wrapper for calling Script.run(). This performs error handling and provides a hook for committing changes. It + exists outside the Script class to ensure it cannot be overridden by a script author. + """ + + class Meta: + # An explicit job name is not set because it doesn't make sense in this context. Currently, there's no scenario + # where jobs other than this one are used. Therefore, it is hidden, resulting in a cleaner job table overview. + name = '' + + def run_script(self, script, request, data, commit): + """ + Core script execution task. We capture this within a method to allow for conditionally wrapping it with the + event_tracking context manager (which is bypassed if commit == False). + + Args: + request: The WSGI request associated with this execution (if any) + data: A dictionary of data to be passed to the script upon execution + commit: Passed through to Script.run() + """ + logger = logging.getLogger(f"netbox.scripts.{script.full_name}") + logger.info(f"Running script (commit={commit})") + + try: + try: + with transaction.atomic(): + script.output = script.run(data, commit) + if not commit: + raise AbortTransaction() + except AbortTransaction: + script.log_info(message=_("Database changes have been reverted automatically.")) + if script.failed: + logger.warning(f"Script failed") + raise + + except Exception as e: + if type(e) is AbortScript: + msg = _("Script aborted with error: ") + str(e) + if is_report(type(script)): + script.log_failure(message=msg) + else: + script.log_failure(msg) + logger.error(f"Script aborted with error: {e}") + + else: + stacktrace = traceback.format_exc() + script.log_failure( + message=_("An exception occurred: ") + f"`{type(e).__name__}: {e}`\n```\n{stacktrace}\n```" + ) + logger.error(f"Exception raised during script execution: {e}") + + if type(e) is not AbortTransaction: + script.log_info(message=_("Database changes have been reverted due to error.")) + + # Clear all pending events. Job termination (including setting the status) is handled by the job framework. + if request: + clear_events.send(request) + raise + + # Update the job data regardless of the execution status of the job. Successes should be reported as well as + # failures. + finally: + self.job.data = script.get_job_data() + + def run(self, data, request=None, commit=True, **kwargs): + """ + Run the script. + + Args: + job: The Job associated with this execution + data: A dictionary of data to be passed to the script upon execution + request: The WSGI request associated with this execution (if any) + commit: Passed through to Script.run() + """ + script = ScriptModel.objects.get(pk=self.job.object_id).python_class() + + # Add files to form data + if request: + files = request.FILES + for field_name, fileobj in files.items(): + data[field_name] = fileobj + + # Add the current request as a property of the script + script.request = request + + # Execute the script. If commit is True, wrap it with the event_tracking context manager to ensure we process + # change logging, event rules, etc. + with event_tracking(request) if commit else nullcontext(): + self.run_script(script, request, data, commit) diff --git a/netbox/extras/management/commands/runscript.py b/netbox/extras/management/commands/runscript.py index dbfbb40d9..b6d6810ac 100644 --- a/netbox/extras/management/commands/runscript.py +++ b/netbox/extras/management/commands/runscript.py @@ -1,19 +1,14 @@ import json import logging import sys -import traceback import uuid from django.contrib.auth import get_user_model from django.core.management.base import BaseCommand, CommandError -from django.db import transaction +from django.utils.module_loading import import_string -from core.choices import JobStatusChoices -from core.models import Job +from extras.jobs import ScriptJob from extras.scripts import get_module_and_script -from extras.signals import clear_events -from netbox.context_managers import event_tracking -from utilities.exceptions import AbortTransaction from utilities.request import NetBoxFakeRequest @@ -33,44 +28,6 @@ class Command(BaseCommand): parser.add_argument('script', help="Script to run") def handle(self, *args, **options): - - def _run_script(): - """ - Core script execution task. We capture this within a subfunction to allow for conditionally wrapping it with - the event_tracking context manager (which is bypassed if commit == False). - """ - try: - try: - with transaction.atomic(): - script.output = script.run(data=data, commit=commit) - if not commit: - raise AbortTransaction() - except AbortTransaction: - script.log_info("Database changes have been reverted automatically.") - clear_events.send(request) - job.data = script.get_job_data() - job.terminate() - except Exception as e: - stacktrace = traceback.format_exc() - script.log_failure( - f"An exception occurred: `{type(e).__name__}: {e}`\n```\n{stacktrace}\n```" - ) - script.log_info("Database changes have been reverted due to error.") - logger.error(f"Exception raised during script execution: {e}") - clear_events.send(request) - job.data = script.get_job_data() - job.terminate(status=JobStatusChoices.STATUS_ERRORED, error=repr(e)) - - # Print any test method results - for test_name, attrs in job.data['tests'].items(): - self.stdout.write( - "\t{}: {} success, {} info, {} warning, {} failure".format( - test_name, attrs['success'], attrs['info'], attrs['warning'], attrs['failure'] - ) - ) - - logger.info(f"Script completed in {job.duration}") - User = get_user_model() # Params @@ -84,8 +41,8 @@ class Command(BaseCommand): data = {} module_name, script_name = script.split('.', 1) - module, script = get_module_and_script(module_name, script_name) - script = script.python_class + module, script_obj = get_module_and_script(module_name, script_name) + script = script_obj.python_class # Take user from command line if provided and exists, other if options['user']: @@ -120,40 +77,29 @@ class Command(BaseCommand): # Initialize the script form script = script() form = script.as_form(data, None) - - # Create the job - job = Job.objects.create( - object=module, - name=script.class_name, - user=User.objects.filter(is_superuser=True).order_by('pk')[0], - job_id=uuid.uuid4() - ) - - request = NetBoxFakeRequest({ - 'META': {}, - 'POST': data, - 'GET': {}, - 'FILES': {}, - 'user': user, - 'path': '', - 'id': job.job_id - }) - - if form.is_valid(): - job.status = JobStatusChoices.STATUS_RUNNING - job.save() - - logger.info(f"Running script (commit={commit})") - script.request = request - - # Execute the script. If commit is True, wrap it with the event_tracking context manager to ensure we process - # change logging, webhooks, etc. - with event_tracking(request): - _run_script() - else: + if not form.is_valid(): logger.error('Data is not valid:') for field, errors in form.errors.get_json_data().items(): for error in errors: logger.error(f'\t{field}: {error.get("message")}') - job.status = JobStatusChoices.STATUS_ERRORED - job.save() + raise CommandError() + + # Execute the script. + job = ScriptJob.enqueue( + instance=script_obj, + user=user, + immediate=True, + data=data, + request=NetBoxFakeRequest({ + 'META': {}, + 'POST': data, + 'GET': {}, + 'FILES': {}, + 'user': user, + 'path': '', + 'id': uuid.uuid4() + }), + commit=commit, + ) + + logger.info(f"Script completed in {job.duration}") diff --git a/netbox/extras/migrations/0120_customfield_related_object_filter.py b/netbox/extras/migrations/0120_customfield_related_object_filter.py new file mode 100644 index 000000000..431ce5bf9 --- /dev/null +++ b/netbox/extras/migrations/0120_customfield_related_object_filter.py @@ -0,0 +1,18 @@ +# Generated by Django 5.0.7 on 2024-07-26 01:49 + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('extras', '0119_eventrule_event_types'), + ] + + operations = [ + migrations.AddField( + model_name='customfield', + name='related_object_filter', + field=models.JSONField(blank=True, null=True), + ), + ] diff --git a/netbox/extras/models/customfields.py b/netbox/extras/models/customfields.py index 1d84a3f4f..79ba75098 100644 --- a/netbox/extras/models/customfields.py +++ b/netbox/extras/models/customfields.py @@ -154,6 +154,14 @@ class CustomField(CloningMixin, ExportTemplatesMixin, ChangeLoggedModel): 'Default value for the field (must be a JSON value). Encapsulate strings with double quotes (e.g. "Foo").' ) ) + related_object_filter = models.JSONField( + blank=True, + null=True, + help_text=_( + 'Filter the object selection choices using a query_params dict (must be a JSON value).' + 'Encapsulate strings with double quotes (e.g. "Foo").' + ) + ) weight = models.PositiveSmallIntegerField( default=100, verbose_name=_('display weight'), @@ -373,6 +381,17 @@ class CustomField(CloningMixin, ExportTemplatesMixin, ChangeLoggedModel): .format(type=self.get_type_display()) }) + # Related object filter can be set only for object-type fields, and must contain a dictionary mapping (if set) + if self.related_object_filter is not None: + if self.type not in (CustomFieldTypeChoices.TYPE_OBJECT, CustomFieldTypeChoices.TYPE_MULTIOBJECT): + raise ValidationError({ + 'related_object_filter': _("A related object filter can be defined only for object fields.") + }) + if type(self.related_object_filter) is not dict: + raise ValidationError({ + 'related_object_filter': _("Filter must be defined as a dictionary mapping attributes to values.") + }) + def serialize(self, value): """ Prepare a value for storage as JSON data. @@ -511,7 +530,8 @@ class CustomField(CloningMixin, ExportTemplatesMixin, ChangeLoggedModel): field = field_class( queryset=model.objects.all(), required=required, - initial=initial + initial=initial, + query_params=self.related_object_filter ) # Multiple objects @@ -522,6 +542,7 @@ class CustomField(CloningMixin, ExportTemplatesMixin, ChangeLoggedModel): queryset=model.objects.all(), required=required, initial=initial, + query_params=self.related_object_filter ) # Text diff --git a/netbox/extras/models/notifications.py b/netbox/extras/models/notifications.py index dba059ea7..a15dfe9aa 100644 --- a/netbox/extras/models/notifications.py +++ b/netbox/extras/models/notifications.py @@ -27,7 +27,7 @@ def get_event_type_choices(): """ return [ (name, event.text) - for name, event in registry['events'].items() + for name, event in registry['event_types'].items() ] @@ -102,7 +102,7 @@ class Notification(models.Model): """ Returns the registered Event which triggered this Notification. """ - return registry['events'].get(self.event_type) + return registry['event_types'].get(self.event_type) class NotificationGroup(ChangeLoggedModel): diff --git a/netbox/extras/scripts.py b/netbox/extras/scripts.py index f6cc2bad0..96d08d8af 100644 --- a/netbox/extras/scripts.py +++ b/netbox/extras/scripts.py @@ -2,32 +2,23 @@ import inspect import json import logging import os -import traceback -from datetime import timedelta import yaml from django import forms from django.conf import settings from django.core.validators import RegexValidator -from django.db import transaction from django.utils import timezone from django.utils.functional import classproperty from django.utils.translation import gettext as _ -from core.choices import JobStatusChoices -from core.models import Job from extras.choices import LogLevelChoices -from extras.models import ScriptModule, Script as ScriptModel -from extras.signals import clear_events +from extras.models import ScriptModule from ipam.formfields import IPAddressFormField, IPNetworkFormField from ipam.validators import MaxPrefixLengthValidator, MinPrefixLengthValidator, prefix_validator -from netbox.context_managers import event_tracking -from utilities.exceptions import AbortScript, AbortTransaction from utilities.forms import add_blank_choice from utilities.forms.fields import DynamicModelChoiceField, DynamicModelMultipleChoiceField from utilities.forms.widgets import DatePicker, DateTimePicker from .forms import ScriptForm -from .utils import is_report __all__ = ( @@ -48,7 +39,6 @@ __all__ = ( 'StringVar', 'TextVar', 'get_module_and_script', - 'run_script', ) @@ -613,111 +603,3 @@ def get_module_and_script(module_name, script_name): module = ScriptModule.objects.get(file_path=f'{module_name}.py') script = module.scripts.get(name=script_name) return module, script - - -def run_script(data, job, request=None, commit=True, **kwargs): - """ - A wrapper for calling Script.run(). This performs error handling and provides a hook for committing changes. It - exists outside the Script class to ensure it cannot be overridden by a script author. - - Args: - data: A dictionary of data to be passed to the script upon execution - job: The Job associated with this execution - request: The WSGI request associated with this execution (if any) - commit: Passed through to Script.run() - """ - job.start() - - script = ScriptModel.objects.get(pk=job.object_id).python_class() - - logger = logging.getLogger(f"netbox.scripts.{script.full_name}") - logger.info(f"Running script (commit={commit})") - - # Add files to form data - if request: - files = request.FILES - for field_name, fileobj in files.items(): - data[field_name] = fileobj - - # Add the current request as a property of the script - script.request = request - - def set_job_data(script): - job.data = { - 'log': script.messages, - 'output': script.output, - 'tests': script.tests, - } - - return job - - def _run_script(job): - """ - Core script execution task. We capture this within a subfunction to allow for conditionally wrapping it with - the event_tracking context manager (which is bypassed if commit == False). - """ - try: - try: - with transaction.atomic(): - script.output = script.run(data, commit) - if not commit: - raise AbortTransaction() - except AbortTransaction: - script.log_info(message=_("Database changes have been reverted automatically.")) - if request: - clear_events.send(request) - - job.data = script.get_job_data() - if script.failed: - logger.warning(f"Script failed") - job.terminate(status=JobStatusChoices.STATUS_FAILED) - else: - job.terminate() - - except Exception as e: - if type(e) is AbortScript: - msg = _("Script aborted with error: ") + str(e) - if is_report(type(script)): - script.log_failure(message=msg) - else: - script.log_failure(msg) - - logger.error(f"Script aborted with error: {e}") - else: - stacktrace = traceback.format_exc() - script.log_failure( - message=_("An exception occurred: ") + f"`{type(e).__name__}: {e}`\n```\n{stacktrace}\n```" - ) - logger.error(f"Exception raised during script execution: {e}") - script.log_info(message=_("Database changes have been reverted due to error.")) - - job.data = script.get_job_data() - job.terminate(status=JobStatusChoices.STATUS_ERRORED, error=repr(e)) - if request: - clear_events.send(request) - - logger.info(f"Script completed in {job.duration}") - - # Execute the script. If commit is True, wrap it with the event_tracking context manager to ensure we process - # change logging, event rules, etc. - if commit: - with event_tracking(request): - _run_script(job) - else: - _run_script(job) - - # Schedule the next job if an interval has been set - if job.interval: - new_scheduled_time = job.scheduled + timedelta(minutes=job.interval) - Job.enqueue( - run_script, - instance=job.object, - name=job.name, - user=job.user, - schedule_at=new_scheduled_time, - interval=job.interval, - job_timeout=script.job_timeout, - data=data, - request=request, - commit=commit - ) diff --git a/netbox/extras/tests/test_filtersets.py b/netbox/extras/tests/test_filtersets.py index 144dec5d0..9048d5fd9 100644 --- a/netbox/extras/tests/test_filtersets.py +++ b/netbox/extras/tests/test_filtersets.py @@ -23,7 +23,7 @@ from virtualization.models import Cluster, ClusterGroup, ClusterType class CustomFieldTestCase(TestCase, ChangeLoggedFilterSetTests): queryset = CustomField.objects.all() filterset = CustomFieldFilterSet - ignore_fields = ('default',) + ignore_fields = ('default', 'related_object_filter') @classmethod def setUpTestData(cls): diff --git a/netbox/extras/views.py b/netbox/extras/views.py index 2ade79c96..8eb9f412d 100644 --- a/netbox/extras/views.py +++ b/netbox/extras/views.py @@ -6,6 +6,7 @@ from django.db.models import Count, Q from django.http import HttpResponseBadRequest, HttpResponseForbidden, HttpResponse from django.shortcuts import get_object_or_404, redirect, render from django.urls import reverse +from django.utils.module_loading import import_string from django.utils import timezone from django.utils.translation import gettext as _ from django.views.generic import View @@ -35,7 +36,6 @@ from virtualization.models import VirtualMachine from . import filtersets, forms, tables from .constants import LOG_LEVEL_RANK from .models import * -from .scripts import run_script from .tables import ReportResultsTable, ScriptResultsTable @@ -551,14 +551,6 @@ class EventRuleListView(generic.ObjectListView): class EventRuleView(generic.ObjectView): queryset = EventRule.objects.all() - def get_extra_context(self, request, instance): - return { - 'event_types': [ - event for name, event in registry['events'].items() - if name in instance.event_types - ] - } - @register_model_view(EventRule, 'edit') class EventRuleEditView(generic.ObjectEditView): @@ -1175,10 +1167,9 @@ class ScriptView(BaseScriptView): if not get_workers_for_queue('default'): messages.error(request, _("Unable to run script: RQ worker process not running.")) elif form.is_valid(): - job = Job.enqueue( - run_script, + ScriptJob = import_string("extras.jobs.ScriptJob") + job = ScriptJob.enqueue( instance=script, - name=script_class.class_name, user=request.user, schedule_at=form.cleaned_data.pop('_schedule_at'), interval=form.cleaned_data.pop('_interval'), @@ -1246,7 +1237,10 @@ class ScriptResultView(TableMixin, generic.ObjectView): table = None index = 0 - log_threshold = LOG_LEVEL_RANK.get(request.GET.get('log_threshold', LogLevelChoices.LOG_DEFAULT)) + try: + log_threshold = LOG_LEVEL_RANK[request.GET.get('log_threshold', LogLevelChoices.LOG_DEBUG)] + except KeyError: + log_threshold = LOG_LEVEL_RANK[LogLevelChoices.LOG_DEBUG] if job.data: if 'log' in job.data: @@ -1303,12 +1297,16 @@ class ScriptResultView(TableMixin, generic.ObjectView): if job.completed: table = self.get_table(job, request, bulk_actions=False) + log_threshold = request.GET.get('log_threshold', LogLevelChoices.LOG_DEBUG) + if log_threshold not in LOG_LEVEL_RANK: + log_threshold = LogLevelChoices.LOG_DEBUG + context = { 'script': job.object, 'job': job, 'table': table, 'log_levels': dict(LogLevelChoices), - 'log_threshold': request.GET.get('log_threshold', LogLevelChoices.LOG_DEFAULT) + 'log_threshold': log_threshold, } if job.data and 'log' in job.data: diff --git a/netbox/ipam/utils.py b/netbox/ipam/utils.py index ccf6cb632..3297abd8f 100644 --- a/netbox/ipam/utils.py +++ b/netbox/ipam/utils.py @@ -90,42 +90,45 @@ def add_available_ipaddresses(prefix, ipaddress_list, is_pool=False): return output -def available_vlans_from_range(vlans, vlan_group, vlan_range): +def available_vlans_from_range(vlans, vlan_group, vid_range): """ Create fake records for all gaps between used VLANs """ - min_vid = int(vlan_range.lower) if vlan_range else VLAN_VID_MIN - max_vid = int(vlan_range.upper) if vlan_range else VLAN_VID_MAX + min_vid = int(vid_range.lower) if vid_range else VLAN_VID_MIN + max_vid = int(vid_range.upper) if vid_range else VLAN_VID_MAX if not vlans: return [{ 'vid': min_vid, 'vlan_group': vlan_group, - 'available': max_vid - min_vid + 1 + 'available': max_vid - min_vid }] - prev_vid = max_vid + prev_vid = min_vid - 1 new_vlans = [] for vlan in vlans: + + # Ignore VIDs outside the range + if not min_vid <= vlan.vid < max_vid: + continue + + # Annotate any available VIDs between the previous (or minimum) VID + # and the current VID if vlan.vid - prev_vid > 1: new_vlans.append({ 'vid': prev_vid + 1, 'vlan_group': vlan_group, 'available': vlan.vid - prev_vid - 1, }) + prev_vid = vlan.vid - if vlans[0].vid > min_vid: - new_vlans.append({ - 'vid': min_vid, - 'vlan_group': vlan_group, - 'available': vlans[0].vid - min_vid, - }) + # Annotate any remaining available VLANs if prev_vid < max_vid: new_vlans.append({ 'vid': prev_vid + 1, 'vlan_group': vlan_group, - 'available': max_vid - prev_vid, + 'available': max_vid - prev_vid - 1, }) return new_vlans @@ -136,8 +139,8 @@ def add_available_vlans(vlans, vlan_group): Create fake records for all gaps between used VLANs """ new_vlans = [] - for vlan_range in vlan_group.vid_ranges: - new_vlans.extend(available_vlans_from_range(vlans, vlan_group, vlan_range)) + for vid_range in vlan_group.vid_ranges: + new_vlans.extend(available_vlans_from_range(vlans, vlan_group, vid_range)) vlans = list(vlans) + new_vlans vlans.sort(key=lambda v: v.vid if type(v) is VLAN else v['vid']) diff --git a/netbox/netbox/constants.py b/netbox/netbox/constants.py index cdeda583d..b8c679ec0 100644 --- a/netbox/netbox/constants.py +++ b/netbox/netbox/constants.py @@ -23,6 +23,9 @@ ADVISORY_LOCK_KEYS = { 'wirelesslangroup': 105600, 'inventoryitem': 105700, 'inventoryitemtemplate': 105800, + + # Jobs + 'job-schedules': 110100, } # Default view action permission mapping diff --git a/netbox/netbox/events.py b/netbox/netbox/events.py index ee2561720..2fa4b6e78 100644 --- a/netbox/netbox/events.py +++ b/netbox/netbox/events.py @@ -2,41 +2,41 @@ from dataclasses import dataclass from netbox.registry import registry -EVENT_TYPE_INFO = 'info' -EVENT_TYPE_SUCCESS = 'success' -EVENT_TYPE_WARNING = 'warning' -EVENT_TYPE_DANGER = 'danger' +EVENT_TYPE_KIND_INFO = 'info' +EVENT_TYPE_KIND_SUCCESS = 'success' +EVENT_TYPE_KIND_WARNING = 'warning' +EVENT_TYPE_KIND_DANGER = 'danger' __all__ = ( - 'EVENT_TYPE_DANGER', - 'EVENT_TYPE_INFO', - 'EVENT_TYPE_SUCCESS', - 'EVENT_TYPE_WARNING', - 'Event', - 'get_event', + 'EVENT_TYPE_KIND_DANGER', + 'EVENT_TYPE_KIND_INFO', + 'EVENT_TYPE_KIND_SUCCESS', + 'EVENT_TYPE_KIND_WARNING', + 'EventType', + 'get_event_type', 'get_event_type_choices', 'get_event_text', ) -def get_event(name): - return registry['events'].get(name) +def get_event_type(name): + return registry['event_types'].get(name) def get_event_text(name): - if event := registry['events'].get(name): + if event := registry['event_types'].get(name): return event.text return '' def get_event_type_choices(): return [ - (event.name, event.text) for event in registry['events'].values() + (event.name, event.text) for event in registry['event_types'].values() ] @dataclass -class Event: +class EventType: """ A type of event which can occur in NetBox. Event rules can be defined to automatically perform some action in response to an event. @@ -44,32 +44,32 @@ class Event: Args: name: The unique name under which the event is registered. text: The human-friendly event name. This should support translation. - type: The event's classification (info, success, warning, or danger). The default type is info. + kind: The event's classification (info, success, warning, or danger). The default type is info. """ name: str text: str - type: str = EVENT_TYPE_INFO + kind: str = EVENT_TYPE_KIND_INFO def __str__(self): return self.text def register(self): - if self.name in registry['events']: - raise Exception(f"An event named {self.name} has already been registered!") - registry['events'][self.name] = self + if self.name in registry['event_types']: + raise Exception(f"An event type named {self.name} has already been registered!") + registry['event_types'][self.name] = self def color(self): return { - EVENT_TYPE_INFO: 'blue', - EVENT_TYPE_SUCCESS: 'green', - EVENT_TYPE_WARNING: 'orange', - EVENT_TYPE_DANGER: 'red', - }.get(self.type) + EVENT_TYPE_KIND_INFO: 'blue', + EVENT_TYPE_KIND_SUCCESS: 'green', + EVENT_TYPE_KIND_WARNING: 'orange', + EVENT_TYPE_KIND_DANGER: 'red', + }.get(self.kind) def icon(self): return { - EVENT_TYPE_INFO: 'mdi mdi-information', - EVENT_TYPE_SUCCESS: 'mdi mdi-check-circle', - EVENT_TYPE_WARNING: 'mdi mdi-alert-box', - EVENT_TYPE_DANGER: 'mdi mdi-alert-octagon', - }.get(self.type) + EVENT_TYPE_KIND_INFO: 'mdi mdi-information', + EVENT_TYPE_KIND_SUCCESS: 'mdi mdi-check-circle', + EVENT_TYPE_KIND_WARNING: 'mdi mdi-alert-box', + EVENT_TYPE_KIND_DANGER: 'mdi mdi-alert-octagon', + }.get(self.kind) diff --git a/netbox/netbox/models/features.py b/netbox/netbox/models/features.py index b270382d3..15957b547 100644 --- a/netbox/netbox/models/features.py +++ b/netbox/netbox/models/features.py @@ -289,7 +289,7 @@ class CustomFieldsMixin(models.Model): # Validate uniqueness if enforced if custom_fields[field_name].validation_unique and value not in CUSTOMFIELD_EMPTY_VALUES: - if self._meta.model.objects.filter(**{ + if self._meta.model.objects.exclude(pk=self.pk).filter(**{ f'custom_field_data__{field_name}': value }).exists(): raise ValidationError(_("Custom field '{name}' must have a unique value.").format( diff --git a/netbox/netbox/registry.py b/netbox/netbox/registry.py index 44cdfb92b..0920cbccf 100644 --- a/netbox/netbox/registry.py +++ b/netbox/netbox/registry.py @@ -25,7 +25,7 @@ registry = Registry({ 'counter_fields': collections.defaultdict(dict), 'data_backends': dict(), 'denormalized_fields': collections.defaultdict(list), - 'events': dict(), + 'event_types': dict(), 'model_features': dict(), 'models': collections.defaultdict(set), 'plugins': dict(), diff --git a/netbox/templates/circuits/circuitgroupassignment.html b/netbox/templates/circuits/circuitgroupassignment.html index 333fa9d89..bd277a5fc 100644 --- a/netbox/templates/circuits/circuitgroupassignment.html +++ b/netbox/templates/circuits/circuitgroupassignment.html @@ -20,23 +20,23 @@ - + - + - +
{% trans "Group" %}{{ object.group }}{{ object.group|linkify }}
{% trans "Circuit" %}{{ object.circuit }}{{ object.circuit|linkify }}
{% trans "Priority" %}{{ object.priority }}{{ object.get_priority_display }}
{% include 'inc/panels/tags.html' %} - {% include 'inc/panels/custom_fields.html' %} {% plugin_left_page object %}
+ {% include 'inc/panels/custom_fields.html' %} {% plugin_right_page object %}
diff --git a/netbox/templates/core/plugin.html b/netbox/templates/core/plugin.html index 71a55dd79..32269d9f6 100644 --- a/netbox/templates/core/plugin.html +++ b/netbox/templates/core/plugin.html @@ -32,7 +32,7 @@ {% trans "Overview" %} - {% if True or not plugin.is_local and 'commercial' not in settings.RELEASE.features %} + {% if not plugin.is_local and not settings.RELEASE.features.commercial %}