Merge branch 'feature' into 8356-vm-virtual-disk

This commit is contained in:
Jeremy Stretch 2023-11-01 15:22:30 -04:00
commit 4ae3f43b6a
98 changed files with 13331 additions and 259 deletions

View File

@ -87,3 +87,24 @@ The following colors are supported:
* `gray` * `gray`
* `black` * `black`
* `white` * `white`
---
## PROTECTION_RULES
!!! tip "Dynamic Configuration Parameter"
This is a mapping of models to [custom validators](../customization/custom-validation.md) against which an object is evaluated immediately prior to its deletion. If validation fails, the object is not deleted. An example is provided below:
```python
PROTECTION_RULES = {
"dcim.site": [
{
"status": {
"eq": "decommissioning"
}
},
"my_plugin.validators.Validator1",
]
}
```

View File

@ -26,6 +26,8 @@ The `CustomValidator` class supports several validation types:
* `regex`: Application of a [regular expression](https://en.wikipedia.org/wiki/Regular_expression) * `regex`: Application of a [regular expression](https://en.wikipedia.org/wiki/Regular_expression)
* `required`: A value must be specified * `required`: A value must be specified
* `prohibited`: A value must _not_ be specified * `prohibited`: A value must _not_ be specified
* `eq`: A value must be equal to the specified value
* `neq`: A value must _not_ be equal to the specified value
The `min` and `max` types should be defined for numeric values, whereas `min_length`, `max_length`, and `regex` are suitable for character strings (text values). The `required` and `prohibited` validators may be used for any field, and should be passed a value of `True`. The `min` and `max` types should be defined for numeric values, whereas `min_length`, `max_length`, and `regex` are suitable for character strings (text values). The `required` and `prohibited` validators may be used for any field, and should be passed a value of `True`.

View File

@ -77,6 +77,9 @@ If selected, this component will be treated as if a cable has been connected.
Virtual interfaces can be bound to a physical parent interface. This is helpful for modeling virtual interfaces which employ encapsulation on a physical interface, such as an 802.1Q VLAN-tagged subinterface. Virtual interfaces can be bound to a physical parent interface. This is helpful for modeling virtual interfaces which employ encapsulation on a physical interface, such as an 802.1Q VLAN-tagged subinterface.
!!! note
An interface with one or more child interfaces assigned cannot be deleted until all its child interfaces have been deleted or reassigned.
### Bridged Interface ### Bridged Interface
Interfaces can be bridged to other interfaces on a device in two manners: symmetric or grouped. Interfaces can be bridged to other interfaces on a device in two manners: symmetric or grouped.

View File

@ -16,6 +16,9 @@ The interface's name. Must be unique to the assigned VM.
Identifies the parent interface of a subinterface (e.g. used to employ encapsulation). Identifies the parent interface of a subinterface (e.g. used to employ encapsulation).
!!! note
An interface with one or more child interfaces assigned cannot be deleted until all its child interfaces have been deleted or reassigned.
### Bridged Interface ### Bridged Interface
An interface on the same VM with which this interface is bridged. An interface on the same VM with which this interface is bridged.

View File

@ -0,0 +1,23 @@
# Data Backends
[Data sources](../../models/core/datasource.md) can be defined to reference data which exists on systems of record outside NetBox, such as a git repository or Amazon S3 bucket. Plugins can register their own backend classes to introduce support for additional resource types. This is done by subclassing NetBox's `DataBackend` class.
```python title="data_backends.py"
from netbox.data_backends import DataBackend
class MyDataBackend(DataBackend):
name = 'mybackend'
label = 'My Backend'
...
```
To register one or more data backends with NetBox, define a list named `backends` at the end of this file:
```python title="data_backends.py"
backends = [MyDataBackend]
```
!!! tip
The path to the list of search indexes can be modified by setting `data_backends` in the PluginConfig instance.
::: core.data_backends.DataBackend

View File

@ -109,6 +109,7 @@ NetBox looks for the `config` variable within a plugin's `__init__.py` to load i
| `middleware` | A list of middleware classes to append after NetBox's build-in middleware | | `middleware` | A list of middleware classes to append after NetBox's build-in middleware |
| `queues` | A list of custom background task queues to create | | `queues` | A list of custom background task queues to create |
| `search_extensions` | The dotted path to the list of search index classes (default: `search.indexes`) | | `search_extensions` | The dotted path to the list of search index classes (default: `search.indexes`) |
| `data_backends` | The dotted path to the list of data source backend classes (default: `data_backends.backends`) |
| `template_extensions` | The dotted path to the list of template extension classes (default: `template_content.template_extensions`) | | `template_extensions` | The dotted path to the list of template extension classes (default: `template_content.template_extensions`) |
| `menu_items` | The dotted path to the list of menu items provided by the plugin (default: `navigation.menu_items`) | | `menu_items` | The dotted path to the list of menu items provided by the plugin (default: `navigation.menu_items`) |
| `graphql_schema` | The dotted path to the plugin's GraphQL schema class, if any (default: `graphql.schema`) | | `graphql_schema` | The dotted path to the plugin's GraphQL schema class, if any (default: `graphql.schema`) |

View File

@ -136,6 +136,7 @@ nav:
- Forms: 'plugins/development/forms.md' - Forms: 'plugins/development/forms.md'
- Filters & Filter Sets: 'plugins/development/filtersets.md' - Filters & Filter Sets: 'plugins/development/filtersets.md'
- Search: 'plugins/development/search.md' - Search: 'plugins/development/search.md'
- Data Backends: 'plugins/development/data-backends.md'
- REST API: 'plugins/development/rest-api.md' - REST API: 'plugins/development/rest-api.md'
- GraphQL API: 'plugins/development/graphql-api.md' - GraphQL API: 'plugins/development/graphql-api.md'
- Background Tasks: 'plugins/development/background-tasks.md' - Background Tasks: 'plugins/development/background-tasks.md'

View File

@ -85,7 +85,7 @@ class CircuitTypeSerializer(NetBoxModelSerializer):
class Meta: class Meta:
model = CircuitType model = CircuitType
fields = [ fields = [
'id', 'url', 'display', 'name', 'slug', 'description', 'tags', 'custom_fields', 'created', 'last_updated', 'id', 'url', 'display', 'name', 'slug', 'color', 'description', 'tags', 'custom_fields', 'created', 'last_updated',
'circuit_count', 'circuit_count',
] ]

View File

@ -137,7 +137,7 @@ class CircuitTypeFilterSet(OrganizationalModelFilterSet):
class Meta: class Meta:
model = CircuitType model = CircuitType
fields = ['id', 'name', 'slug', 'description'] fields = ['id', 'name', 'slug', 'color', 'description']
class CircuitFilterSet(NetBoxModelFilterSet, TenancyFilterSet, ContactModelFilterSet): class CircuitFilterSet(NetBoxModelFilterSet, TenancyFilterSet, ContactModelFilterSet):
@ -154,12 +154,12 @@ class CircuitFilterSet(NetBoxModelFilterSet, TenancyFilterSet, ContactModelFilte
provider_account_id = django_filters.ModelMultipleChoiceFilter( provider_account_id = django_filters.ModelMultipleChoiceFilter(
field_name='provider_account', field_name='provider_account',
queryset=ProviderAccount.objects.all(), queryset=ProviderAccount.objects.all(),
label=_('ProviderAccount (ID)'), label=_('Provider account (ID)'),
) )
provider_network_id = django_filters.ModelMultipleChoiceFilter( provider_network_id = django_filters.ModelMultipleChoiceFilter(
field_name='terminations__provider_network', field_name='terminations__provider_network',
queryset=ProviderNetwork.objects.all(), queryset=ProviderNetwork.objects.all(),
label=_('ProviderNetwork (ID)'), label=_('Provider network (ID)'),
) )
type_id = django_filters.ModelMultipleChoiceFilter( type_id = django_filters.ModelMultipleChoiceFilter(
queryset=CircuitType.objects.all(), queryset=CircuitType.objects.all(),

View File

@ -7,7 +7,7 @@ from ipam.models import ASN
from netbox.forms import NetBoxModelBulkEditForm from netbox.forms import NetBoxModelBulkEditForm
from tenancy.models import Tenant from tenancy.models import Tenant
from utilities.forms import add_blank_choice from utilities.forms import add_blank_choice
from utilities.forms.fields import CommentField, DynamicModelChoiceField, DynamicModelMultipleChoiceField from utilities.forms.fields import ColorField, CommentField, DynamicModelChoiceField, DynamicModelMultipleChoiceField
from utilities.forms.widgets import DatePicker, NumberWithOptions from utilities.forms.widgets import DatePicker, NumberWithOptions
__all__ = ( __all__ = (
@ -91,6 +91,10 @@ class ProviderNetworkBulkEditForm(NetBoxModelBulkEditForm):
class CircuitTypeBulkEditForm(NetBoxModelBulkEditForm): class CircuitTypeBulkEditForm(NetBoxModelBulkEditForm):
color = ColorField(
label=_('Color'),
required=False
)
description = forms.CharField( description = forms.CharField(
label=_('Description'), label=_('Description'),
max_length=200, max_length=200,
@ -99,9 +103,9 @@ class CircuitTypeBulkEditForm(NetBoxModelBulkEditForm):
model = CircuitType model = CircuitType
fieldsets = ( fieldsets = (
(None, ('description',)), (None, ('color', 'description')),
) )
nullable_fields = ('description',) nullable_fields = ('color', 'description')
class CircuitBulkEditForm(NetBoxModelBulkEditForm): class CircuitBulkEditForm(NetBoxModelBulkEditForm):

View File

@ -3,6 +3,7 @@ from django import forms
from circuits.choices import CircuitStatusChoices from circuits.choices import CircuitStatusChoices
from circuits.models import * from circuits.models import *
from dcim.models import Site from dcim.models import Site
from django.utils.safestring import mark_safe
from django.utils.translation import gettext_lazy as _ from django.utils.translation import gettext_lazy as _
from netbox.forms import NetBoxModelImportForm from netbox.forms import NetBoxModelImportForm
from tenancy.models import Tenant from tenancy.models import Tenant
@ -64,7 +65,10 @@ class CircuitTypeImportForm(NetBoxModelImportForm):
class Meta: class Meta:
model = CircuitType model = CircuitType
fields = ('name', 'slug', 'description', 'tags') fields = ('name', 'slug', 'color', 'description', 'tags')
help_texts = {
'color': mark_safe(_('RGB color in hexadecimal. Example:') + ' <code>00ff00</code>'),
}
class CircuitImportForm(NetBoxModelImportForm): class CircuitImportForm(NetBoxModelImportForm):

View File

@ -7,7 +7,7 @@ from dcim.models import Region, Site, SiteGroup
from ipam.models import ASN from ipam.models import ASN
from netbox.forms import NetBoxModelFilterSetForm from netbox.forms import NetBoxModelFilterSetForm
from tenancy.forms import TenancyFilterForm, ContactModelFilterForm from tenancy.forms import TenancyFilterForm, ContactModelFilterForm
from utilities.forms.fields import DynamicModelMultipleChoiceField, TagFilterField from utilities.forms.fields import ColorField, DynamicModelMultipleChoiceField, TagFilterField
from utilities.forms.widgets import DatePicker, NumberWithOptions from utilities.forms.widgets import DatePicker, NumberWithOptions
__all__ = ( __all__ = (
@ -88,7 +88,7 @@ class ProviderNetworkFilterForm(NetBoxModelFilterSetForm):
label=_('Provider') label=_('Provider')
) )
service_id = forms.CharField( service_id = forms.CharField(
label=_('Service id'), label=_('Service ID'),
max_length=100, max_length=100,
required=False required=False
) )
@ -97,8 +97,17 @@ class ProviderNetworkFilterForm(NetBoxModelFilterSetForm):
class CircuitTypeFilterForm(NetBoxModelFilterSetForm): class CircuitTypeFilterForm(NetBoxModelFilterSetForm):
model = CircuitType model = CircuitType
fieldsets = (
(None, ('q', 'filter_id', 'tag')),
(_('Attributes'), ('color',)),
)
tag = TagFilterField(model) tag = TagFilterField(model)
color = ColorField(
label=_('Color'),
required=False
)
class CircuitFilterForm(TenancyFilterForm, ContactModelFilterForm, NetBoxModelFilterSetForm): class CircuitFilterForm(TenancyFilterForm, ContactModelFilterForm, NetBoxModelFilterSetForm):
model = Circuit model = Circuit

View File

@ -76,14 +76,14 @@ class CircuitTypeForm(NetBoxModelForm):
fieldsets = ( fieldsets = (
(_('Circuit Type'), ( (_('Circuit Type'), (
'name', 'slug', 'description', 'tags', 'name', 'slug', 'color', 'description', 'tags',
)), )),
) )
class Meta: class Meta:
model = CircuitType model = CircuitType
fields = [ fields = [
'name', 'slug', 'description', 'tags', 'name', 'slug', 'color', 'description', 'tags',
] ]

View File

@ -0,0 +1,18 @@
# Generated by Django 4.2.5 on 2023-10-20 21:25
from django.db import migrations
import utilities.fields
class Migration(migrations.Migration):
dependencies = [
('circuits', '0042_provideraccount'),
]
operations = [
migrations.AddField(
model_name='circuittype',
name='color',
field=utilities.fields.ColorField(blank=True, max_length=6),
),
]

View File

@ -7,6 +7,7 @@ from circuits.choices import *
from dcim.models import CabledObjectModel from dcim.models import CabledObjectModel
from netbox.models import ChangeLoggedModel, OrganizationalModel, PrimaryModel from netbox.models import ChangeLoggedModel, OrganizationalModel, PrimaryModel
from netbox.models.features import ContactsMixin, CustomFieldsMixin, CustomLinksMixin, ImageAttachmentsMixin, TagsMixin from netbox.models.features import ContactsMixin, CustomFieldsMixin, CustomLinksMixin, ImageAttachmentsMixin, TagsMixin
from utilities.fields import ColorField
__all__ = ( __all__ = (
'Circuit', 'Circuit',
@ -20,6 +21,11 @@ class CircuitType(OrganizationalModel):
Circuits can be organized by their functional role. For example, a user might wish to define CircuitTypes named Circuits can be organized by their functional role. For example, a user might wish to define CircuitTypes named
"Long Haul," "Metro," or "Out-of-Band". "Long Haul," "Metro," or "Out-of-Band".
""" """
color = ColorField(
verbose_name=_('color'),
blank=True
)
def get_absolute_url(self): def get_absolute_url(self):
return reverse('circuits:circuittype', args=[self.pk]) return reverse('circuits:circuittype', args=[self.pk])

View File

@ -28,6 +28,7 @@ class CircuitTypeTable(NetBoxTable):
linkify=True, linkify=True,
verbose_name=_('Name'), verbose_name=_('Name'),
) )
color = columns.ColorColumn()
tags = columns.TagColumn( tags = columns.TagColumn(
url_name='circuits:circuittype_list' url_name='circuits:circuittype_list'
) )
@ -40,7 +41,7 @@ class CircuitTypeTable(NetBoxTable):
class Meta(NetBoxTable.Meta): class Meta(NetBoxTable.Meta):
model = CircuitType model = CircuitType
fields = ( fields = (
'pk', 'id', 'name', 'circuit_count', 'description', 'slug', 'tags', 'created', 'last_updated', 'actions', 'pk', 'id', 'name', 'circuit_count', 'color', 'description', 'slug', 'tags', 'created', 'last_updated', 'actions',
) )
default_columns = ('pk', 'name', 'circuit_count', 'description', 'slug') default_columns = ('pk', 'name', 'circuit_count', 'description', 'slug')

View File

@ -4,6 +4,7 @@ from core.choices import *
from core.models import * from core.models import *
from netbox.api.fields import ChoiceField, ContentTypeField from netbox.api.fields import ChoiceField, ContentTypeField
from netbox.api.serializers import BaseModelSerializer, NetBoxModelSerializer from netbox.api.serializers import BaseModelSerializer, NetBoxModelSerializer
from netbox.utils import get_data_backend_choices
from users.api.nested_serializers import NestedUserSerializer from users.api.nested_serializers import NestedUserSerializer
from .nested_serializers import * from .nested_serializers import *
@ -19,7 +20,7 @@ class DataSourceSerializer(NetBoxModelSerializer):
view_name='core-api:datasource-detail' view_name='core-api:datasource-detail'
) )
type = ChoiceField( type = ChoiceField(
choices=DataSourceTypeChoices choices=get_data_backend_choices()
) )
status = ChoiceField( status = ChoiceField(
choices=DataSourceStatusChoices, choices=DataSourceStatusChoices,
@ -68,5 +69,5 @@ class JobSerializer(BaseModelSerializer):
model = Job model = Job
fields = [ fields = [
'id', 'url', 'display', 'object_type', 'object_id', 'name', 'status', 'created', 'scheduled', 'interval', 'id', 'url', 'display', 'object_type', 'object_id', 'name', 'status', 'created', 'scheduled', 'interval',
'started', 'completed', 'user', 'data', 'job_id', 'started', 'completed', 'user', 'data', 'error', 'job_id',
] ]

View File

@ -7,18 +7,6 @@ from utilities.choices import ChoiceSet
# Data sources # Data sources
# #
class DataSourceTypeChoices(ChoiceSet):
LOCAL = 'local'
GIT = 'git'
AMAZON_S3 = 'amazon-s3'
CHOICES = (
(LOCAL, _('Local'), 'gray'),
(GIT, 'Git', 'blue'),
(AMAZON_S3, 'Amazon S3', 'blue'),
)
class DataSourceStatusChoices(ChoiceSet): class DataSourceStatusChoices(ChoiceSet):
NEW = 'new' NEW = 'new'
QUEUED = 'queued' QUEUED = 'queued'

View File

@ -10,61 +10,24 @@ from django import forms
from django.conf import settings from django.conf import settings
from django.utils.translation import gettext as _ from django.utils.translation import gettext as _
from netbox.registry import registry from netbox.data_backends import DataBackend
from .choices import DataSourceTypeChoices from netbox.utils import register_data_backend
from .exceptions import SyncError from .exceptions import SyncError
__all__ = ( __all__ = (
'LocalBackend',
'GitBackend', 'GitBackend',
'LocalBackend',
'S3Backend', 'S3Backend',
) )
logger = logging.getLogger('netbox.data_backends') logger = logging.getLogger('netbox.data_backends')
def register_backend(name): @register_data_backend()
"""
Decorator for registering a DataBackend class.
"""
def _wrapper(cls):
registry['data_backends'][name] = cls
return cls
return _wrapper
class DataBackend:
parameters = {}
sensitive_parameters = []
# Prevent Django's template engine from calling the backend
# class when referenced via DataSource.backend_class
do_not_call_in_templates = True
def __init__(self, url, **kwargs):
self.url = url
self.params = kwargs
self.config = self.init_config()
def init_config(self):
"""
Hook to initialize the instance's configuration.
"""
return
@property
def url_scheme(self):
return urlparse(self.url).scheme.lower()
@contextmanager
def fetch(self):
raise NotImplemented()
@register_backend(DataSourceTypeChoices.LOCAL)
class LocalBackend(DataBackend): class LocalBackend(DataBackend):
name = 'local'
label = _('Local')
is_local = True
@contextmanager @contextmanager
def fetch(self): def fetch(self):
@ -74,8 +37,10 @@ class LocalBackend(DataBackend):
yield local_path yield local_path
@register_backend(DataSourceTypeChoices.GIT) @register_data_backend()
class GitBackend(DataBackend): class GitBackend(DataBackend):
name = 'git'
label = 'Git'
parameters = { parameters = {
'username': forms.CharField( 'username': forms.CharField(
required=False, required=False,
@ -144,8 +109,10 @@ class GitBackend(DataBackend):
local_path.cleanup() local_path.cleanup()
@register_backend(DataSourceTypeChoices.AMAZON_S3) @register_data_backend()
class S3Backend(DataBackend): class S3Backend(DataBackend):
name = 'amazon-s3'
label = 'Amazon S3'
parameters = { parameters = {
'aws_access_key_id': forms.CharField( 'aws_access_key_id': forms.CharField(
label=_('AWS access key ID'), label=_('AWS access key ID'),

View File

@ -4,6 +4,7 @@ from django.utils.translation import gettext as _
import django_filters import django_filters
from netbox.filtersets import BaseFilterSet, ChangeLoggedModelFilterSet, NetBoxModelFilterSet from netbox.filtersets import BaseFilterSet, ChangeLoggedModelFilterSet, NetBoxModelFilterSet
from netbox.utils import get_data_backend_choices
from .choices import * from .choices import *
from .models import * from .models import *
@ -16,7 +17,7 @@ __all__ = (
class DataSourceFilterSet(NetBoxModelFilterSet): class DataSourceFilterSet(NetBoxModelFilterSet):
type = django_filters.MultipleChoiceFilter( type = django_filters.MultipleChoiceFilter(
choices=DataSourceTypeChoices, choices=get_data_backend_choices,
null_value=None null_value=None
) )
status = django_filters.MultipleChoiceFilter( status = django_filters.MultipleChoiceFilter(

View File

@ -1,10 +1,9 @@
from django import forms from django import forms
from django.utils.translation import gettext_lazy as _ from django.utils.translation import gettext_lazy as _
from core.choices import DataSourceTypeChoices
from core.models import * from core.models import *
from netbox.forms import NetBoxModelBulkEditForm from netbox.forms import NetBoxModelBulkEditForm
from utilities.forms import add_blank_choice from netbox.utils import get_data_backend_choices
from utilities.forms.fields import CommentField from utilities.forms.fields import CommentField
from utilities.forms.widgets import BulkEditNullBooleanSelect from utilities.forms.widgets import BulkEditNullBooleanSelect
@ -16,9 +15,8 @@ __all__ = (
class DataSourceBulkEditForm(NetBoxModelBulkEditForm): class DataSourceBulkEditForm(NetBoxModelBulkEditForm):
type = forms.ChoiceField( type = forms.ChoiceField(
label=_('Type'), label=_('Type'),
choices=add_blank_choice(DataSourceTypeChoices), choices=get_data_backend_choices,
required=False, required=False
initial=''
) )
enabled = forms.NullBooleanField( enabled = forms.NullBooleanField(
required=False, required=False,

View File

@ -8,6 +8,7 @@ from core.models import *
from extras.forms.mixins import SavedFiltersMixin from extras.forms.mixins import SavedFiltersMixin
from extras.utils import FeatureQuery from extras.utils import FeatureQuery
from netbox.forms import NetBoxModelFilterSetForm from netbox.forms import NetBoxModelFilterSetForm
from netbox.utils import get_data_backend_choices
from utilities.forms import BOOLEAN_WITH_BLANK_CHOICES, FilterForm from utilities.forms import BOOLEAN_WITH_BLANK_CHOICES, FilterForm
from utilities.forms.fields import ContentTypeChoiceField, DynamicModelMultipleChoiceField from utilities.forms.fields import ContentTypeChoiceField, DynamicModelMultipleChoiceField
from utilities.forms.widgets import APISelectMultiple, DateTimePicker from utilities.forms.widgets import APISelectMultiple, DateTimePicker
@ -27,7 +28,7 @@ class DataSourceFilterForm(NetBoxModelFilterSetForm):
) )
type = forms.MultipleChoiceField( type = forms.MultipleChoiceField(
label=_('Type'), label=_('Type'),
choices=DataSourceTypeChoices, choices=get_data_backend_choices,
required=False required=False
) )
status = forms.MultipleChoiceField( status = forms.MultipleChoiceField(

View File

@ -7,6 +7,7 @@ from core.forms.mixins import SyncedDataMixin
from core.models import * from core.models import *
from netbox.forms import NetBoxModelForm from netbox.forms import NetBoxModelForm
from netbox.registry import registry from netbox.registry import registry
from netbox.utils import get_data_backend_choices
from utilities.forms import get_field_value from utilities.forms import get_field_value
from utilities.forms.fields import CommentField from utilities.forms.fields import CommentField
from utilities.forms.widgets import HTMXSelect from utilities.forms.widgets import HTMXSelect
@ -18,6 +19,10 @@ __all__ = (
class DataSourceForm(NetBoxModelForm): class DataSourceForm(NetBoxModelForm):
type = forms.ChoiceField(
choices=get_data_backend_choices,
widget=HTMXSelect()
)
comments = CommentField() comments = CommentField()
class Meta: class Meta:
@ -26,7 +31,6 @@ class DataSourceForm(NetBoxModelForm):
'name', 'type', 'source_url', 'enabled', 'description', 'comments', 'ignore_rules', 'tags', 'name', 'type', 'source_url', 'enabled', 'description', 'comments', 'ignore_rules', 'tags',
] ]
widgets = { widgets = {
'type': HTMXSelect(),
'ignore_rules': forms.Textarea( 'ignore_rules': forms.Textarea(
attrs={ attrs={
'rows': 5, 'rows': 5,
@ -56,6 +60,7 @@ class DataSourceForm(NetBoxModelForm):
# Add backend-specific form fields # Add backend-specific form fields
self.backend_fields = [] self.backend_fields = []
if backend:
for name, form_field in backend.parameters.items(): for name, form_field in backend.parameters.items():
field_name = f'backend_{name}' field_name = f'backend_{name}'
self.backend_fields.append(field_name) self.backend_fields.append(field_name)

View File

@ -25,7 +25,7 @@ def sync_datasource(job, *args, **kwargs):
job.terminate() job.terminate()
except Exception as e: except Exception as e:
job.terminate(status=JobStatusChoices.STATUS_ERRORED) job.terminate(status=JobStatusChoices.STATUS_ERRORED, error=str(e))
DataSource.objects.filter(pk=datasource.pk).update(status=DataSourceStatusChoices.FAILED) DataSource.objects.filter(pk=datasource.pk).update(status=DataSourceStatusChoices.FAILED)
if type(e) in (SyncError, JobTimeoutException): if type(e) in (SyncError, JobTimeoutException):
logging.error(e) logging.error(e)

View File

@ -0,0 +1,18 @@
# Generated by Django 4.2.6 on 2023-10-20 17:47
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0005_job_created_auto_now'),
]
operations = [
migrations.AlterField(
model_name='datasource',
name='type',
field=models.CharField(max_length=50),
),
]

View File

@ -0,0 +1,18 @@
# Generated by Django 4.2.6 on 2023-10-23 20:28
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0006_datasource_type_remove_choices'),
]
operations = [
migrations.AddField(
model_name='job',
name='error',
field=models.TextField(blank=True, editable=False),
),
]

View File

@ -45,9 +45,7 @@ class DataSource(JobsMixin, PrimaryModel):
) )
type = models.CharField( type = models.CharField(
verbose_name=_('type'), verbose_name=_('type'),
max_length=50, max_length=50
choices=DataSourceTypeChoices,
default=DataSourceTypeChoices.LOCAL
) )
source_url = models.CharField( source_url = models.CharField(
max_length=200, max_length=200,
@ -96,8 +94,9 @@ class DataSource(JobsMixin, PrimaryModel):
def docs_url(self): def docs_url(self):
return f'{settings.STATIC_URL}docs/models/{self._meta.app_label}/{self._meta.model_name}/' return f'{settings.STATIC_URL}docs/models/{self._meta.app_label}/{self._meta.model_name}/'
def get_type_color(self): def get_type_display(self):
return DataSourceTypeChoices.colors.get(self.type) if backend := registry['data_backends'].get(self.type):
return backend.label
def get_status_color(self): def get_status_color(self):
return DataSourceStatusChoices.colors.get(self.status) return DataSourceStatusChoices.colors.get(self.status)
@ -110,10 +109,6 @@ class DataSource(JobsMixin, PrimaryModel):
def backend_class(self): def backend_class(self):
return registry['data_backends'].get(self.type) return registry['data_backends'].get(self.type)
@property
def is_local(self):
return self.type == DataSourceTypeChoices.LOCAL
@property @property
def ready_for_sync(self): def ready_for_sync(self):
return self.enabled and self.status not in ( return self.enabled and self.status not in (
@ -123,8 +118,14 @@ class DataSource(JobsMixin, PrimaryModel):
def clean(self): def clean(self):
# Validate data backend type
if self.type and self.type not in registry['data_backends']:
raise ValidationError({
'type': _("Unknown backend type: {type}".format(type=self.type))
})
# Ensure URL scheme matches selected type # Ensure URL scheme matches selected type
if self.type == DataSourceTypeChoices.LOCAL and self.url_scheme not in ('file', ''): if self.backend_class.is_local and self.url_scheme not in ('file', ''):
raise ValidationError({ raise ValidationError({
'source_url': f"URLs for local sources must start with file:// (or specify no scheme)" 'source_url': f"URLs for local sources must start with file:// (or specify no scheme)"
}) })

View File

@ -92,6 +92,11 @@ class Job(models.Model):
null=True, null=True,
blank=True blank=True
) )
error = models.TextField(
verbose_name=_('error'),
editable=False,
blank=True
)
job_id = models.UUIDField( job_id = models.UUIDField(
verbose_name=_('job ID'), verbose_name=_('job ID'),
unique=True unique=True
@ -158,7 +163,7 @@ class Job(models.Model):
# Handle webhooks # Handle webhooks
self.trigger_webhooks(event=EVENT_JOB_START) self.trigger_webhooks(event=EVENT_JOB_START)
def terminate(self, status=JobStatusChoices.STATUS_COMPLETED): def terminate(self, status=JobStatusChoices.STATUS_COMPLETED, error=None):
""" """
Mark the job as completed, optionally specifying a particular termination status. Mark the job as completed, optionally specifying a particular termination status.
""" """
@ -168,6 +173,8 @@ class Job(models.Model):
# Mark the job as completed # Mark the job as completed
self.status = status self.status = status
if error:
self.error = error
self.completed = timezone.now() self.completed = timezone.now()
self.save() self.save()

View File

@ -0,0 +1,20 @@
import django_tables2 as tables
from netbox.registry import registry
__all__ = (
'BackendTypeColumn',
)
class BackendTypeColumn(tables.Column):
"""
Display a data backend type.
"""
def render(self, value):
if backend := registry['data_backends'].get(value):
return backend.label
return value
def value(self, value):
return value

View File

@ -3,6 +3,7 @@ import django_tables2 as tables
from core.models import * from core.models import *
from netbox.tables import NetBoxTable, columns from netbox.tables import NetBoxTable, columns
from .columns import BackendTypeColumn
__all__ = ( __all__ = (
'DataFileTable', 'DataFileTable',
@ -15,8 +16,8 @@ class DataSourceTable(NetBoxTable):
verbose_name=_('Name'), verbose_name=_('Name'),
linkify=True linkify=True
) )
type = columns.ChoiceFieldColumn( type = BackendTypeColumn(
verbose_name=_('Type'), verbose_name=_('Type')
) )
status = columns.ChoiceFieldColumn( status = columns.ChoiceFieldColumn(
verbose_name=_('Status'), verbose_name=_('Status'),
@ -34,8 +35,8 @@ class DataSourceTable(NetBoxTable):
class Meta(NetBoxTable.Meta): class Meta(NetBoxTable.Meta):
model = DataSource model = DataSource
fields = ( fields = (
'pk', 'id', 'name', 'type', 'status', 'enabled', 'source_url', 'description', 'comments', 'parameters', 'created', 'pk', 'id', 'name', 'type', 'status', 'enabled', 'source_url', 'description', 'comments', 'parameters',
'last_updated', 'file_count', 'created', 'last_updated', 'file_count',
) )
default_columns = ('pk', 'name', 'type', 'status', 'enabled', 'description', 'file_count') default_columns = ('pk', 'name', 'type', 'status', 'enabled', 'description', 'file_count')

View File

@ -47,7 +47,7 @@ class JobTable(NetBoxTable):
model = Job model = Job
fields = ( fields = (
'pk', 'id', 'object_type', 'object', 'name', 'status', 'created', 'scheduled', 'interval', 'started', 'pk', 'id', 'object_type', 'object', 'name', 'status', 'created', 'scheduled', 'interval', 'started',
'completed', 'user', 'job_id', 'completed', 'user', 'error', 'job_id',
) )
default_columns = ( default_columns = (
'pk', 'id', 'object_type', 'object', 'name', 'status', 'created', 'started', 'completed', 'user', 'pk', 'id', 'object_type', 'object', 'name', 'status', 'created', 'started', 'completed', 'user',

View File

@ -2,7 +2,6 @@ from django.urls import reverse
from django.utils import timezone from django.utils import timezone
from utilities.testing import APITestCase, APIViewTestCases from utilities.testing import APITestCase, APIViewTestCases
from ..choices import *
from ..models import * from ..models import *
@ -26,26 +25,26 @@ class DataSourceTest(APIViewTestCases.APIViewTestCase):
@classmethod @classmethod
def setUpTestData(cls): def setUpTestData(cls):
data_sources = ( data_sources = (
DataSource(name='Data Source 1', type=DataSourceTypeChoices.LOCAL, source_url='file:///var/tmp/source1/'), DataSource(name='Data Source 1', type='local', source_url='file:///var/tmp/source1/'),
DataSource(name='Data Source 2', type=DataSourceTypeChoices.LOCAL, source_url='file:///var/tmp/source2/'), DataSource(name='Data Source 2', type='local', source_url='file:///var/tmp/source2/'),
DataSource(name='Data Source 3', type=DataSourceTypeChoices.LOCAL, source_url='file:///var/tmp/source3/'), DataSource(name='Data Source 3', type='local', source_url='file:///var/tmp/source3/'),
) )
DataSource.objects.bulk_create(data_sources) DataSource.objects.bulk_create(data_sources)
cls.create_data = [ cls.create_data = [
{ {
'name': 'Data Source 4', 'name': 'Data Source 4',
'type': DataSourceTypeChoices.GIT, 'type': 'git',
'source_url': 'https://example.com/git/source4' 'source_url': 'https://example.com/git/source4'
}, },
{ {
'name': 'Data Source 5', 'name': 'Data Source 5',
'type': DataSourceTypeChoices.GIT, 'type': 'git',
'source_url': 'https://example.com/git/source5' 'source_url': 'https://example.com/git/source5'
}, },
{ {
'name': 'Data Source 6', 'name': 'Data Source 6',
'type': DataSourceTypeChoices.GIT, 'type': 'git',
'source_url': 'https://example.com/git/source6' 'source_url': 'https://example.com/git/source6'
}, },
] ]
@ -63,7 +62,7 @@ class DataFileTest(
def setUpTestData(cls): def setUpTestData(cls):
datasource = DataSource.objects.create( datasource = DataSource.objects.create(
name='Data Source 1', name='Data Source 1',
type=DataSourceTypeChoices.LOCAL, type='local',
source_url='file:///var/tmp/source1/' source_url='file:///var/tmp/source1/'
) )

View File

@ -18,21 +18,21 @@ class DataSourceTestCase(TestCase, ChangeLoggedFilterSetTests):
data_sources = ( data_sources = (
DataSource( DataSource(
name='Data Source 1', name='Data Source 1',
type=DataSourceTypeChoices.LOCAL, type='local',
source_url='file:///var/tmp/source1/', source_url='file:///var/tmp/source1/',
status=DataSourceStatusChoices.NEW, status=DataSourceStatusChoices.NEW,
enabled=True enabled=True
), ),
DataSource( DataSource(
name='Data Source 2', name='Data Source 2',
type=DataSourceTypeChoices.LOCAL, type='local',
source_url='file:///var/tmp/source2/', source_url='file:///var/tmp/source2/',
status=DataSourceStatusChoices.SYNCING, status=DataSourceStatusChoices.SYNCING,
enabled=True enabled=True
), ),
DataSource( DataSource(
name='Data Source 3', name='Data Source 3',
type=DataSourceTypeChoices.GIT, type='git',
source_url='https://example.com/git/source3', source_url='https://example.com/git/source3',
status=DataSourceStatusChoices.COMPLETED, status=DataSourceStatusChoices.COMPLETED,
enabled=False enabled=False
@ -45,7 +45,7 @@ class DataSourceTestCase(TestCase, ChangeLoggedFilterSetTests):
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2) self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
def test_type(self): def test_type(self):
params = {'type': [DataSourceTypeChoices.LOCAL]} params = {'type': ['local']}
self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2) self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
def test_enabled(self): def test_enabled(self):
@ -66,9 +66,9 @@ class DataFileTestCase(TestCase, ChangeLoggedFilterSetTests):
@classmethod @classmethod
def setUpTestData(cls): def setUpTestData(cls):
data_sources = ( data_sources = (
DataSource(name='Data Source 1', type=DataSourceTypeChoices.LOCAL, source_url='file:///var/tmp/source1/'), DataSource(name='Data Source 1', type='local', source_url='file:///var/tmp/source1/'),
DataSource(name='Data Source 2', type=DataSourceTypeChoices.LOCAL, source_url='file:///var/tmp/source2/'), DataSource(name='Data Source 2', type='local', source_url='file:///var/tmp/source2/'),
DataSource(name='Data Source 3', type=DataSourceTypeChoices.LOCAL, source_url='file:///var/tmp/source3/'), DataSource(name='Data Source 3', type='local', source_url='file:///var/tmp/source3/'),
) )
DataSource.objects.bulk_create(data_sources) DataSource.objects.bulk_create(data_sources)

View File

@ -1,7 +1,6 @@
from django.utils import timezone from django.utils import timezone
from utilities.testing import ViewTestCases, create_tags from utilities.testing import ViewTestCases, create_tags
from ..choices import *
from ..models import * from ..models import *
@ -11,9 +10,9 @@ class DataSourceTestCase(ViewTestCases.PrimaryObjectViewTestCase):
@classmethod @classmethod
def setUpTestData(cls): def setUpTestData(cls):
data_sources = ( data_sources = (
DataSource(name='Data Source 1', type=DataSourceTypeChoices.LOCAL, source_url='file:///var/tmp/source1/'), DataSource(name='Data Source 1', type='local', source_url='file:///var/tmp/source1/'),
DataSource(name='Data Source 2', type=DataSourceTypeChoices.LOCAL, source_url='file:///var/tmp/source2/'), DataSource(name='Data Source 2', type='local', source_url='file:///var/tmp/source2/'),
DataSource(name='Data Source 3', type=DataSourceTypeChoices.LOCAL, source_url='file:///var/tmp/source3/'), DataSource(name='Data Source 3', type='local', source_url='file:///var/tmp/source3/'),
) )
DataSource.objects.bulk_create(data_sources) DataSource.objects.bulk_create(data_sources)
@ -21,7 +20,7 @@ class DataSourceTestCase(ViewTestCases.PrimaryObjectViewTestCase):
cls.form_data = { cls.form_data = {
'name': 'Data Source X', 'name': 'Data Source X',
'type': DataSourceTypeChoices.GIT, 'type': 'git',
'source_url': 'http:///exmaple/com/foo/bar/', 'source_url': 'http:///exmaple/com/foo/bar/',
'description': 'Something', 'description': 'Something',
'comments': 'Foo bar baz', 'comments': 'Foo bar baz',
@ -29,10 +28,10 @@ class DataSourceTestCase(ViewTestCases.PrimaryObjectViewTestCase):
} }
cls.csv_data = ( cls.csv_data = (
f"name,type,source_url,enabled", "name,type,source_url,enabled",
f"Data Source 4,{DataSourceTypeChoices.LOCAL},file:///var/tmp/source4/,true", "Data Source 4,local,file:///var/tmp/source4/,true",
f"Data Source 5,{DataSourceTypeChoices.LOCAL},file:///var/tmp/source4/,true", "Data Source 5,local,file:///var/tmp/source4/,true",
f"Data Source 6,{DataSourceTypeChoices.GIT},http:///exmaple/com/foo/bar/,false", "Data Source 6,git,http:///exmaple/com/foo/bar/,false",
) )
cls.csv_update_data = ( cls.csv_update_data = (
@ -60,7 +59,7 @@ class DataFileTestCase(
def setUpTestData(cls): def setUpTestData(cls):
datasource = DataSource.objects.create( datasource = DataSource.objects.create(
name='Data Source 1', name='Data Source 1',
type=DataSourceTypeChoices.LOCAL, type='local',
source_url='file:///var/tmp/source1/' source_url='file:///var/tmp/source1/'
) )

View File

@ -343,9 +343,9 @@ class DeviceTypeSerializer(NetBoxModelSerializer):
model = DeviceType model = DeviceType
fields = [ fields = [
'id', 'url', 'display', 'manufacturer', 'default_platform', 'model', 'slug', 'part_number', 'u_height', 'id', 'url', 'display', 'manufacturer', 'default_platform', 'model', 'slug', 'part_number', 'u_height',
'is_full_depth', 'subdevice_role', 'airflow', 'weight', 'weight_unit', 'front_image', 'rear_image', 'exclude_from_utilization', 'is_full_depth', 'subdevice_role', 'airflow', 'weight', 'weight_unit',
'description', 'comments', 'tags', 'custom_fields', 'created', 'last_updated', 'device_count', 'front_image', 'rear_image', 'description', 'comments', 'tags', 'custom_fields', 'created', 'last_updated',
'console_port_template_count', 'console_server_port_template_count', 'power_port_template_count', 'device_count', 'console_port_template_count', 'console_server_port_template_count', 'power_port_template_count',
'power_outlet_template_count', 'interface_template_count', 'front_port_template_count', 'power_outlet_template_count', 'interface_template_count', 'front_port_template_count',
'rear_port_template_count', 'device_bay_template_count', 'module_bay_template_count', 'rear_port_template_count', 'device_bay_template_count', 'module_bay_template_count',
'inventory_item_template_count', 'inventory_item_template_count',

View File

@ -24,6 +24,7 @@ from netbox.api.viewsets import NetBoxModelViewSet, MPTTLockedMixin
from netbox.api.viewsets.mixins import SequentialBulkCreatesMixin from netbox.api.viewsets.mixins import SequentialBulkCreatesMixin
from netbox.constants import NESTED_SERIALIZER_PREFIX from netbox.constants import NESTED_SERIALIZER_PREFIX
from utilities.api import get_serializer_for_model from utilities.api import get_serializer_for_model
from utilities.query_functions import CollateAsChar
from utilities.utils import count_related from utilities.utils import count_related
from virtualization.models import VirtualMachine from virtualization.models import VirtualMachine
from . import serializers from . import serializers
@ -505,6 +506,10 @@ class InterfaceViewSet(PathEndpointMixin, NetBoxModelViewSet):
filterset_class = filtersets.InterfaceFilterSet filterset_class = filtersets.InterfaceFilterSet
brief_prefetch_fields = ['device'] brief_prefetch_fields = ['device']
def get_bulk_destroy_queryset(self):
# Ensure child interfaces are deleted prior to their parents
return self.get_queryset().order_by('device', 'parent', CollateAsChar('_name'))
class FrontPortViewSet(PassThroughPortMixin, NetBoxModelViewSet): class FrontPortViewSet(PassThroughPortMixin, NetBoxModelViewSet):
queryset = FrontPort.objects.prefetch_related( queryset = FrontPort.objects.prefetch_related(

View File

@ -80,10 +80,10 @@ class RackWidthChoices(ChoiceSet):
WIDTH_23IN = 23 WIDTH_23IN = 23
CHOICES = ( CHOICES = (
(WIDTH_10IN, _('10 inches')), (WIDTH_10IN, _('{n} inches').format(n=10)),
(WIDTH_19IN, _('19 inches')), (WIDTH_19IN, _('{n} inches').format(n=19)),
(WIDTH_21IN, _('21 inches')), (WIDTH_21IN, _('{n} inches').format(n=21)),
(WIDTH_23IN, _('23 inches')), (WIDTH_23IN, _('{n} inches').format(n=23)),
) )

View File

@ -496,7 +496,8 @@ class DeviceTypeFilterSet(NetBoxModelFilterSet):
class Meta: class Meta:
model = DeviceType model = DeviceType
fields = [ fields = [
'id', 'model', 'slug', 'part_number', 'u_height', 'is_full_depth', 'subdevice_role', 'airflow', 'weight', 'weight_unit', 'id', 'model', 'slug', 'part_number', 'u_height', 'exclude_from_utilization', 'is_full_depth', 'subdevice_role',
'airflow', 'weight', 'weight_unit',
] ]
def search(self, queryset, name, value): def search(self, queryset, name, value):

View File

@ -420,6 +420,11 @@ class DeviceTypeBulkEditForm(NetBoxModelBulkEditForm):
widget=BulkEditNullBooleanSelect(), widget=BulkEditNullBooleanSelect(),
label=_('Is full depth') label=_('Is full depth')
) )
exclude_from_utilization = forms.NullBooleanField(
required=False,
widget=BulkEditNullBooleanSelect(),
label=_('Exclude from utilization')
)
airflow = forms.ChoiceField( airflow = forms.ChoiceField(
label=_('Airflow'), label=_('Airflow'),
choices=add_blank_choice(DeviceAirflowChoices), choices=add_blank_choice(DeviceAirflowChoices),
@ -445,7 +450,10 @@ class DeviceTypeBulkEditForm(NetBoxModelBulkEditForm):
model = DeviceType model = DeviceType
fieldsets = ( fieldsets = (
(_('Device Type'), ('manufacturer', 'default_platform', 'part_number', 'u_height', 'is_full_depth', 'airflow', 'description')), (_('Device Type'), (
'manufacturer', 'default_platform', 'part_number', 'u_height', 'exclude_from_utilization', 'is_full_depth',
'airflow', 'description',
)),
(_('Weight'), ('weight', 'weight_unit')), (_('Weight'), ('weight', 'weight_unit')),
) )
nullable_fields = ('part_number', 'airflow', 'weight', 'weight_unit', 'description', 'comments') nullable_fields = ('part_number', 'airflow', 'weight', 'weight_unit', 'description', 'comments')

View File

@ -335,8 +335,8 @@ class DeviceTypeImportForm(NetBoxModelImportForm):
class Meta: class Meta:
model = DeviceType model = DeviceType
fields = [ fields = [
'manufacturer', 'default_platform', 'model', 'slug', 'part_number', 'u_height', 'is_full_depth', 'manufacturer', 'default_platform', 'model', 'slug', 'part_number', 'u_height', 'exclude_from_utilization',
'subdevice_role', 'airflow', 'description', 'weight', 'weight_unit', 'comments', 'tags', 'is_full_depth', 'subdevice_role', 'airflow', 'description', 'weight', 'weight_unit', 'comments', 'tags',
] ]

View File

@ -116,17 +116,17 @@ class ModuleCommonForm(forms.Form):
# It is not possible to adopt components already belonging to a module # It is not possible to adopt components already belonging to a module
if adopt_components and existing_item and existing_item.module: if adopt_components and existing_item and existing_item.module:
raise forms.ValidationError( raise forms.ValidationError(
_("Cannot adopt {name} '{resolved_name}' as it already belongs to a module").format( _("Cannot adopt {model} {name} as it already belongs to a module").format(
name=template.component_model.__name__, model=template.component_model.__name__,
resolved_name=resolved_name name=resolved_name
) )
) )
# If we are not adopting components we error if the component exists # If we are not adopting components we error if the component exists
if not adopt_components and resolved_name in installed_components: if not adopt_components and resolved_name in installed_components:
raise forms.ValidationError( raise forms.ValidationError(
_("{name} - {resolved_name} already exists").format( _("A {model} named {name} already exists").format(
name=template.component_model.__name__, model=template.component_model.__name__,
resolved_name=resolved_name name=resolved_name
) )
) )

View File

@ -302,7 +302,8 @@ class DeviceTypeForm(NetBoxModelForm):
fieldsets = ( fieldsets = (
(_('Device Type'), ('manufacturer', 'model', 'slug', 'default_platform', 'description', 'tags')), (_('Device Type'), ('manufacturer', 'model', 'slug', 'default_platform', 'description', 'tags')),
(_('Chassis'), ( (_('Chassis'), (
'u_height', 'is_full_depth', 'part_number', 'subdevice_role', 'airflow', 'weight', 'weight_unit', 'u_height', 'exclude_from_utilization', 'is_full_depth', 'part_number', 'subdevice_role', 'airflow',
'weight', 'weight_unit',
)), )),
(_('Images'), ('front_image', 'rear_image')), (_('Images'), ('front_image', 'rear_image')),
) )
@ -310,9 +311,9 @@ class DeviceTypeForm(NetBoxModelForm):
class Meta: class Meta:
model = DeviceType model = DeviceType
fields = [ fields = [
'manufacturer', 'model', 'slug', 'default_platform', 'part_number', 'u_height', 'is_full_depth', 'manufacturer', 'model', 'slug', 'default_platform', 'part_number', 'u_height', 'exclude_from_utilization',
'subdevice_role', 'airflow', 'weight', 'weight_unit', 'front_image', 'rear_image', 'description', 'is_full_depth', 'subdevice_role', 'airflow', 'weight', 'weight_unit', 'front_image', 'rear_image',
'comments', 'tags', 'description', 'comments', 'tags',
] ]
widgets = { widgets = {
'front_image': ClearableFileInput(attrs={ 'front_image': ClearableFileInput(attrs={

View File

@ -0,0 +1,17 @@
# Generated by Django 4.2.5 on 2023-10-20 22:30
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('dcim', '0181_rename_device_role_device_role'),
]
operations = [
migrations.AddField(
model_name='devicetype',
name='exclude_from_utilization',
field=models.BooleanField(default=False),
),
]

View File

@ -0,0 +1,19 @@
# Generated by Django 4.2.6 on 2023-10-20 11:48
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
dependencies = [
('dcim', '0182_devicetype_exclude_from_utilization'),
]
operations = [
migrations.AlterField(
model_name='interface',
name='parent',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.RESTRICT, related_name='child_interfaces', to='dcim.interface'),
),
]

View File

@ -534,14 +534,16 @@ class FrontPortTemplate(ModularComponentTemplateModel):
# Validate rear port assignment # Validate rear port assignment
if self.rear_port.device_type != self.device_type: if self.rear_port.device_type != self.device_type:
raise ValidationError( raise ValidationError(
_("Rear port ({}) must belong to the same device type").format(self.rear_port) _("Rear port ({name}) must belong to the same device type").format(name=self.rear_port)
) )
# Validate rear port position assignment # Validate rear port position assignment
if self.rear_port_position > self.rear_port.positions: if self.rear_port_position > self.rear_port.positions:
raise ValidationError( raise ValidationError(
_("Invalid rear port position ({}); rear port {} has only {} positions").format( _("Invalid rear port position ({position}); rear port {name} has only {count} positions").format(
self.rear_port_position, self.rear_port.name, self.rear_port.positions position=self.rear_port_position,
name=self.rear_port.name,
count=self.rear_port.positions
) )
) )

View File

@ -537,7 +537,7 @@ class BaseInterface(models.Model):
) )
parent = models.ForeignKey( parent = models.ForeignKey(
to='self', to='self',
on_delete=models.SET_NULL, on_delete=models.RESTRICT,
related_name='child_interfaces', related_name='child_interfaces',
null=True, null=True,
blank=True, blank=True,

View File

@ -106,6 +106,11 @@ class DeviceType(ImageAttachmentsMixin, PrimaryModel, WeightMixin):
default=1.0, default=1.0,
verbose_name=_('height (U)') verbose_name=_('height (U)')
) )
exclude_from_utilization = models.BooleanField(
default=False,
verbose_name=_('exclude from utilization'),
help_text=_('Exclude from rack utilization calculations.')
)
is_full_depth = models.BooleanField( is_full_depth = models.BooleanField(
default=True, default=True,
verbose_name=_('is full depth'), verbose_name=_('is full depth'),
@ -297,8 +302,10 @@ class DeviceType(ImageAttachmentsMixin, PrimaryModel, WeightMixin):
) )
if d.position not in u_available: if d.position not in u_available:
raise ValidationError({ raise ValidationError({
'u_height': _("Device {} in rack {} does not have sufficient space to accommodate a height of " 'u_height': _(
"{}U").format(d, d.rack, self.u_height) "Device {device} in rack {rack} does not have sufficient space to accommodate a "
"height of {height}U"
).format(device=d, rack=d.rack, height=self.u_height)
}) })
# If modifying the height of an existing DeviceType to 0U, check for any instances assigned to a rack position. # If modifying the height of an existing DeviceType to 0U, check for any instances assigned to a rack position.
@ -915,7 +922,7 @@ class Device(
if self.primary_ip4: if self.primary_ip4:
if self.primary_ip4.family != 4: if self.primary_ip4.family != 4:
raise ValidationError({ raise ValidationError({
'primary_ip4': _("{primary_ip4} is not an IPv4 address.").format(primary_ip4=self.primary_ip4) 'primary_ip4': _("{ip} is not an IPv4 address.").format(ip=self.primary_ip4)
}) })
if self.primary_ip4.assigned_object in vc_interfaces: if self.primary_ip4.assigned_object in vc_interfaces:
pass pass
@ -924,13 +931,13 @@ class Device(
else: else:
raise ValidationError({ raise ValidationError({
'primary_ip4': _( 'primary_ip4': _(
"The specified IP address ({primary_ip4}) is not assigned to this device." "The specified IP address ({ip}) is not assigned to this device."
).format(primary_ip4=self.primary_ip4) ).format(ip=self.primary_ip4)
}) })
if self.primary_ip6: if self.primary_ip6:
if self.primary_ip6.family != 6: if self.primary_ip6.family != 6:
raise ValidationError({ raise ValidationError({
'primary_ip6': _("{primary_ip6} is not an IPv6 address.").format(primary_ip6=self.primary_ip6m) 'primary_ip6': _("{ip} is not an IPv6 address.").format(ip=self.primary_ip6)
}) })
if self.primary_ip6.assigned_object in vc_interfaces: if self.primary_ip6.assigned_object in vc_interfaces:
pass pass
@ -939,8 +946,8 @@ class Device(
else: else:
raise ValidationError({ raise ValidationError({
'primary_ip6': _( 'primary_ip6': _(
"The specified IP address ({primary_ip6}) is not assigned to this device." "The specified IP address ({ip}) is not assigned to this device."
).format(primary_ip6=self.primary_ip6) ).format(ip=self.primary_ip6)
}) })
if self.oob_ip: if self.oob_ip:
if self.oob_ip.assigned_object in vc_interfaces: if self.oob_ip.assigned_object in vc_interfaces:
@ -958,17 +965,19 @@ class Device(
raise ValidationError({ raise ValidationError({
'platform': _( 'platform': _(
"The assigned platform is limited to {platform_manufacturer} device types, but this device's " "The assigned platform is limited to {platform_manufacturer} device types, but this device's "
"type belongs to {device_type_manufacturer}." "type belongs to {devicetype_manufacturer}."
).format( ).format(
platform_manufacturer=self.platform.manufacturer, platform_manufacturer=self.platform.manufacturer,
device_type_manufacturer=self.device_type.manufacturer devicetype_manufacturer=self.device_type.manufacturer
) )
}) })
# A Device can only be assigned to a Cluster in the same Site (or no Site) # A Device can only be assigned to a Cluster in the same Site (or no Site)
if self.cluster and self.cluster.site is not None and self.cluster.site != self.site: if self.cluster and self.cluster.site is not None and self.cluster.site != self.site:
raise ValidationError({ raise ValidationError({
'cluster': _("The assigned cluster belongs to a different site ({})").format(self.cluster.site) 'cluster': _("The assigned cluster belongs to a different site ({site})").format(
site=self.cluster.site
)
}) })
# Validate virtual chassis assignment # Validate virtual chassis assignment
@ -1440,8 +1449,8 @@ class VirtualDeviceContext(PrimaryModel):
if primary_ip.family != family: if primary_ip.family != family:
raise ValidationError({ raise ValidationError({
f'primary_ip{family}': _( f'primary_ip{family}': _(
"{primary_ip} is not an IPv{family} address." "{ip} is not an IPv{family} address."
).format(family=family, primary_ip=primary_ip) ).format(family=family, ip=primary_ip)
}) })
device_interfaces = self.device.vc_interfaces(if_master=False) device_interfaces = self.device.vc_interfaces(if_master=False)
if primary_ip.assigned_object not in device_interfaces: if primary_ip.assigned_object not in device_interfaces:

View File

@ -357,7 +357,7 @@ class Rack(ContactsMixin, ImageAttachmentsMixin, PrimaryModel, WeightMixin):
return [u for u in elevation.values()] return [u for u in elevation.values()]
def get_available_units(self, u_height=1, rack_face=None, exclude=None): def get_available_units(self, u_height=1, rack_face=None, exclude=None, ignore_excluded_devices=False):
""" """
Return a list of units within the rack available to accommodate a device of a given U height (default 1). Return a list of units within the rack available to accommodate a device of a given U height (default 1).
Optionally exclude one or more devices when calculating empty units (needed when moving a device from one Optionally exclude one or more devices when calculating empty units (needed when moving a device from one
@ -366,9 +366,13 @@ class Rack(ContactsMixin, ImageAttachmentsMixin, PrimaryModel, WeightMixin):
:param u_height: Minimum number of contiguous free units required :param u_height: Minimum number of contiguous free units required
:param rack_face: The face of the rack (front or rear) required; 'None' if device is full depth :param rack_face: The face of the rack (front or rear) required; 'None' if device is full depth
:param exclude: List of devices IDs to exclude (useful when moving a device within a rack) :param exclude: List of devices IDs to exclude (useful when moving a device within a rack)
:param ignore_excluded_devices: Ignore devices that are marked to exclude from utilization calculations
""" """
# Gather all devices which consume U space within the rack # Gather all devices which consume U space within the rack
devices = self.devices.prefetch_related('device_type').filter(position__gte=1) devices = self.devices.prefetch_related('device_type').filter(position__gte=1)
if ignore_excluded_devices:
devices = devices.exclude(device_type__exclude_from_utilization=True)
if exclude is not None: if exclude is not None:
devices = devices.exclude(pk__in=exclude) devices = devices.exclude(pk__in=exclude)
@ -453,7 +457,7 @@ class Rack(ContactsMixin, ImageAttachmentsMixin, PrimaryModel, WeightMixin):
""" """
# Determine unoccupied units # Determine unoccupied units
total_units = len(list(self.units)) total_units = len(list(self.units))
available_units = self.get_available_units(u_height=0.5) available_units = self.get_available_units(u_height=0.5, ignore_excluded_devices=True)
# Remove reserved units # Remove reserved units
for ru in self.get_reserved_units(): for ru in self.get_reserved_units():
@ -558,9 +562,9 @@ class RackReservation(PrimaryModel):
invalid_units = [u for u in self.units if u not in self.rack.units] invalid_units = [u for u in self.units if u not in self.rack.units]
if invalid_units: if invalid_units:
raise ValidationError({ raise ValidationError({
'units': _("Invalid unit(s) for {}U rack: {}").format( 'units': _("Invalid unit(s) for {height}U rack: {unit_list}").format(
self.rack.u_height, height=self.rack.u_height,
', '.join([str(u) for u in invalid_units]), unit_list=', '.join([str(u) for u in invalid_units])
), ),
}) })
@ -571,8 +575,8 @@ class RackReservation(PrimaryModel):
conflicting_units = [u for u in self.units if u in reserved_units] conflicting_units = [u for u in self.units if u in reserved_units]
if conflicting_units: if conflicting_units:
raise ValidationError({ raise ValidationError({
'units': _('The following units have already been reserved: {}').format( 'units': _('The following units have already been reserved: {unit_list}').format(
', '.join([str(u) for u in conflicting_units]), unit_list=', '.join([str(u) for u in conflicting_units])
) )
}) })

View File

@ -159,6 +159,7 @@ class CableTraceSVG:
labels.append(location_label) labels.append(location_label)
elif instance._meta.model_name == 'circuit': elif instance._meta.model_name == 'circuit':
labels[0] = f'Circuit {instance}' labels[0] = f'Circuit {instance}'
labels.append(instance.type)
labels.append(instance.provider) labels.append(instance.provider)
if instance.description: if instance.description:
labels.append(instance.description) labels.append(instance.description)
@ -181,6 +182,8 @@ class CableTraceSVG:
if hasattr(instance, 'role'): if hasattr(instance, 'role'):
# Device # Device
return instance.role.color return instance.role.color
elif instance._meta.model_name == 'circuit' and instance.type.color:
return instance.type.color
else: else:
# Other parent object # Other parent object
return 'e0e0e0' return 'e0e0e0'

View File

@ -98,6 +98,7 @@ class DeviceTypeTable(NetBoxTable):
verbose_name=_('U Height'), verbose_name=_('U Height'),
template_code='{{ value|floatformat }}' template_code='{{ value|floatformat }}'
) )
exclude_from_utilization = columns.BooleanColumn()
weight = columns.TemplateColumn( weight = columns.TemplateColumn(
verbose_name=_('Weight'), verbose_name=_('Weight'),
template_code=WEIGHT, template_code=WEIGHT,
@ -142,9 +143,9 @@ class DeviceTypeTable(NetBoxTable):
class Meta(NetBoxTable.Meta): class Meta(NetBoxTable.Meta):
model = models.DeviceType model = models.DeviceType
fields = ( fields = (
'pk', 'id', 'model', 'manufacturer', 'default_platform', 'slug', 'part_number', 'u_height', 'is_full_depth', 'pk', 'id', 'model', 'manufacturer', 'default_platform', 'slug', 'part_number', 'u_height',
'subdevice_role', 'airflow', 'weight', 'description', 'comments', 'instance_count', 'tags', 'created', 'exclude_from_utilization', 'is_full_depth', 'subdevice_role', 'airflow', 'weight',
'last_updated', 'description', 'comments', 'instance_count', 'tags', 'created', 'last_updated',
) )
default_columns = ( default_columns = (
'pk', 'model', 'manufacturer', 'part_number', 'u_height', 'is_full_depth', 'instance_count', 'pk', 'model', 'manufacturer', 'part_number', 'u_height', 'is_full_depth', 'instance_count',

View File

@ -1607,6 +1607,33 @@ class InterfaceTest(Mixins.ComponentTraceMixin, APIViewTestCases.APIViewTestCase
}, },
] ]
def test_bulk_delete_child_interfaces(self):
interface1 = Interface.objects.get(name='Interface 1')
device = interface1.device
self.add_permissions('dcim.delete_interface')
# Create a child interface
child = Interface.objects.create(
device=device,
name='Interface 1A',
type=InterfaceTypeChoices.TYPE_VIRTUAL,
parent=interface1
)
self.assertEqual(device.interfaces.count(), 4)
# Attempt to delete only the parent interface
url = self._get_detail_url(interface1)
self.client.delete(url, **self.header)
self.assertEqual(device.interfaces.count(), 4) # Parent was not deleted
# Attempt to bulk delete parent & child together
data = [
{"id": interface1.pk},
{"id": child.pk},
]
self.client.delete(self._get_list_url(), data, format='json', **self.header)
self.assertEqual(device.interfaces.count(), 2) # Child & parent were both deleted
class FrontPortTest(APIViewTestCases.APIViewTestCase): class FrontPortTest(APIViewTestCases.APIViewTestCase):
model = FrontPort model = FrontPort

View File

@ -238,6 +238,40 @@ class RackTestCase(TestCase):
# Check that Device1 is now assigned to Site B # Check that Device1 is now assigned to Site B
self.assertEqual(Device.objects.get(pk=device1.pk).site, site_b) self.assertEqual(Device.objects.get(pk=device1.pk).site, site_b)
def test_utilization(self):
site = Site.objects.first()
rack = Rack.objects.first()
Device(
name='Device 1',
role=DeviceRole.objects.first(),
device_type=DeviceType.objects.first(),
site=site,
rack=rack,
position=1
).save()
rack.refresh_from_db()
self.assertEqual(rack.get_utilization(), 1 / 42 * 100)
# create device excluded from utilization calculations
dt = DeviceType.objects.create(
manufacturer=Manufacturer.objects.first(),
model='Device Type 4',
slug='device-type-4',
u_height=1,
exclude_from_utilization=True
)
Device(
name='Device 2',
role=DeviceRole.objects.first(),
device_type=dt,
site=site,
rack=rack,
position=5
).save()
rack.refresh_from_db()
self.assertEqual(rack.get_utilization(), 1 / 42 * 100)
class DeviceTestCase(TestCase): class DeviceTestCase(TestCase):

View File

@ -2531,6 +2531,36 @@ class InterfaceTestCase(ViewTestCases.DeviceComponentViewTestCase):
response = self.client.get(reverse('dcim:interface_trace', kwargs={'pk': interface1.pk})) response = self.client.get(reverse('dcim:interface_trace', kwargs={'pk': interface1.pk}))
self.assertHttpStatus(response, 200) self.assertHttpStatus(response, 200)
def test_bulk_delete_child_interfaces(self):
interface1 = Interface.objects.get(name='Interface 1')
device = interface1.device
self.add_permissions('dcim.delete_interface')
# Create a child interface
child = Interface.objects.create(
device=device,
name='Interface 1A',
type=InterfaceTypeChoices.TYPE_VIRTUAL,
parent=interface1
)
self.assertEqual(device.interfaces.count(), 6)
# Attempt to delete only the parent interface
data = {
'confirm': True,
}
self.client.post(self._get_url('delete', interface1), data)
self.assertEqual(device.interfaces.count(), 6) # Parent was not deleted
# Attempt to bulk delete parent & child together
data = {
'pk': [interface1.pk, child.pk],
'confirm': True,
'_confirm': True, # Form button
}
self.client.post(self._get_url('bulk_delete'), data)
self.assertEqual(device.interfaces.count(), 4) # Child & parent were both deleted
class FrontPortTestCase(ViewTestCases.DeviceComponentViewTestCase): class FrontPortTestCase(ViewTestCases.DeviceComponentViewTestCase):
model = FrontPort model = FrontPort

View File

@ -1,5 +1,4 @@
import traceback import traceback
from collections import defaultdict
from django.contrib import messages from django.contrib import messages
from django.contrib.contenttypes.models import ContentType from django.contrib.contenttypes.models import ContentType
@ -26,6 +25,7 @@ from tenancy.views import ObjectContactsView
from utilities.forms import ConfirmationForm from utilities.forms import ConfirmationForm
from utilities.paginator import EnhancedPaginator, get_paginate_count from utilities.paginator import EnhancedPaginator, get_paginate_count
from utilities.permissions import get_permission_for_model from utilities.permissions import get_permission_for_model
from utilities.query_functions import CollateAsChar
from utilities.utils import count_related from utilities.utils import count_related
from utilities.views import GetReturnURLMixin, ObjectPermissionRequiredMixin, ViewTab, register_model_view from utilities.views import GetReturnURLMixin, ObjectPermissionRequiredMixin, ViewTab, register_model_view
from virtualization.models import VirtualMachine from virtualization.models import VirtualMachine
@ -2562,7 +2562,8 @@ class InterfaceBulkDisconnectView(BulkDisconnectView):
class InterfaceBulkDeleteView(generic.BulkDeleteView): class InterfaceBulkDeleteView(generic.BulkDeleteView):
queryset = Interface.objects.all() # Ensure child interfaces are deleted prior to their parents
queryset = Interface.objects.order_by('device', 'parent', CollateAsChar('_name'))
filterset = filtersets.InterfaceFilterSet filterset = filtersets.InterfaceFilterSet
table = tables.InterfaceTable table = tables.InterfaceTable

View File

@ -491,7 +491,7 @@ class ConfigRevisionForm(BootstrapMixin, forms.ModelForm, metaclass=ConfigFormMe
(_('Security'), ('ALLOWED_URL_SCHEMES',)), (_('Security'), ('ALLOWED_URL_SCHEMES',)),
(_('Banners'), ('BANNER_LOGIN', 'BANNER_MAINTENANCE', 'BANNER_TOP', 'BANNER_BOTTOM')), (_('Banners'), ('BANNER_LOGIN', 'BANNER_MAINTENANCE', 'BANNER_TOP', 'BANNER_BOTTOM')),
(_('Pagination'), ('PAGINATE_COUNT', 'MAX_PAGE_SIZE')), (_('Pagination'), ('PAGINATE_COUNT', 'MAX_PAGE_SIZE')),
(_('Validation'), ('CUSTOM_VALIDATORS',)), (_('Validation'), ('CUSTOM_VALIDATORS', 'PROTECTION_RULES')),
(_('User Preferences'), ('DEFAULT_USER_PREFERENCES',)), (_('User Preferences'), ('DEFAULT_USER_PREFERENCES',)),
(_('Miscellaneous'), ( (_('Miscellaneous'), (
'MAINTENANCE_MODE', 'GRAPHQL_ENABLED', 'CHANGELOG_RETENTION', 'JOB_RETENTION', 'MAPS_URL', 'MAINTENANCE_MODE', 'GRAPHQL_ENABLED', 'CHANGELOG_RETENTION', 'JOB_RETENTION', 'MAPS_URL',
@ -508,6 +508,7 @@ class ConfigRevisionForm(BootstrapMixin, forms.ModelForm, metaclass=ConfigFormMe
'BANNER_TOP': forms.Textarea(attrs={'class': 'font-monospace'}), 'BANNER_TOP': forms.Textarea(attrs={'class': 'font-monospace'}),
'BANNER_BOTTOM': forms.Textarea(attrs={'class': 'font-monospace'}), 'BANNER_BOTTOM': forms.Textarea(attrs={'class': 'font-monospace'}),
'CUSTOM_VALIDATORS': forms.Textarea(attrs={'class': 'font-monospace'}), 'CUSTOM_VALIDATORS': forms.Textarea(attrs={'class': 'font-monospace'}),
'PROTECTION_RULES': forms.Textarea(attrs={'class': 'font-monospace'}),
'comment': forms.Textarea(), 'comment': forms.Textarea(),
} }

View File

@ -59,7 +59,7 @@ class Command(BaseCommand):
logger.error(f"Exception raised during script execution: {e}") logger.error(f"Exception raised during script execution: {e}")
clear_webhooks.send(request) clear_webhooks.send(request)
job.data = ScriptOutputSerializer(script).data job.data = ScriptOutputSerializer(script).data
job.terminate(status=JobStatusChoices.STATUS_ERRORED) job.terminate(status=JobStatusChoices.STATUS_ERRORED, error=str(e))
logger.info(f"Script completed in {job.duration}") logger.info(f"Script completed in {job.duration}")

View File

@ -0,0 +1,17 @@
# Generated by Django 4.2.6 on 2023-10-30 14:04
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
('extras', '0098_webhook_custom_field_data_webhook_tags'),
]
operations = [
migrations.AlterModelOptions(
name='cachedvalue',
options={'ordering': ('weight', 'object_type', 'value', 'object_id')},
),
]

View File

@ -287,8 +287,8 @@ class CustomField(CloningMixin, ExportTemplatesMixin, ChangeLoggedModel):
except ValidationError as err: except ValidationError as err:
raise ValidationError({ raise ValidationError({
'default': _( 'default': _(
'Invalid default value "{default}": {message}' 'Invalid default value "{value}": {error}'
).format(default=self.default, message=err.message) ).format(value=self.default, error=err.message)
}) })
# Minimum/maximum values can be set only for numeric fields # Minimum/maximum values can be set only for numeric fields
@ -332,8 +332,8 @@ class CustomField(CloningMixin, ExportTemplatesMixin, ChangeLoggedModel):
elif self.object_type: elif self.object_type:
raise ValidationError({ raise ValidationError({
'object_type': _( 'object_type': _(
"{type_display} fields may not define an object type.") "{type} fields may not define an object type.")
.format(type_display=self.get_type_display()) .format(type=self.get_type_display())
}) })
def serialize(self, value): def serialize(self, value):

View File

@ -50,7 +50,7 @@ class CachedValue(models.Model):
) )
class Meta: class Meta:
ordering = ('weight', 'object_type', 'object_id') ordering = ('weight', 'object_type', 'value', 'object_id')
verbose_name = _('cached value') verbose_name = _('cached value')
verbose_name_plural = _('cached values') verbose_name_plural = _('cached values')

View File

@ -40,8 +40,8 @@ def run_report(job, *args, **kwargs):
try: try:
report.run(job) report.run(job)
except Exception: except Exception as e:
job.terminate(status=JobStatusChoices.STATUS_ERRORED) job.terminate(status=JobStatusChoices.STATUS_ERRORED, error=str(e))
logging.error(f"Error during execution of report {job.name}") logging.error(f"Error during execution of report {job.name}")
finally: finally:
# Schedule the next job if an interval has been set # Schedule the next job if an interval has been set
@ -230,7 +230,7 @@ class Report(object):
stacktrace = traceback.format_exc() stacktrace = traceback.format_exc()
self.log_failure(None, f"An exception occurred: {type(e).__name__}: {e} <pre>{stacktrace}</pre>") self.log_failure(None, f"An exception occurred: {type(e).__name__}: {e} <pre>{stacktrace}</pre>")
logger.error(f"Exception raised during report execution: {e}") logger.error(f"Exception raised during report execution: {e}")
job.terminate(status=JobStatusChoices.STATUS_ERRORED) job.terminate(status=JobStatusChoices.STATUS_ERRORED, error=str(e))
# Perform any post-run tasks # Perform any post-run tasks
self.post_run() self.post_run()

View File

@ -519,7 +519,7 @@ def run_script(data, request, job, commit=True, **kwargs):
logger.error(f"Exception raised during script execution: {e}") logger.error(f"Exception raised during script execution: {e}")
script.log_info("Database changes have been reverted due to error.") script.log_info("Database changes have been reverted due to error.")
job.data = ScriptOutputSerializer(script).data job.data = ScriptOutputSerializer(script).data
job.terminate(status=JobStatusChoices.STATUS_ERRORED) job.terminate(status=JobStatusChoices.STATUS_ERRORED, error=str(e))
clear_webhooks.send(request) clear_webhooks.send(request)
logger.info(f"Script completed in {job.duration}") logger.info(f"Script completed in {job.duration}")

View File

@ -2,8 +2,10 @@ import importlib
import logging import logging
from django.contrib.contenttypes.models import ContentType from django.contrib.contenttypes.models import ContentType
from django.core.exceptions import ValidationError
from django.db.models.signals import m2m_changed, post_save, pre_delete from django.db.models.signals import m2m_changed, post_save, pre_delete
from django.dispatch import receiver, Signal from django.dispatch import receiver, Signal
from django.utils.translation import gettext_lazy as _
from django_prometheus.models import model_deletes, model_inserts, model_updates from django_prometheus.models import model_deletes, model_inserts, model_updates
from extras.validators import CustomValidator from extras.validators import CustomValidator
@ -178,11 +180,7 @@ m2m_changed.connect(handle_cf_removed_obj_types, sender=CustomField.content_type
# Custom validation # Custom validation
# #
@receiver(post_clean) def run_validators(instance, validators):
def run_custom_validators(sender, instance, **kwargs):
config = get_config()
model_name = f'{sender._meta.app_label}.{sender._meta.model_name}'
validators = config.CUSTOM_VALIDATORS.get(model_name, [])
for validator in validators: for validator in validators:
@ -198,6 +196,29 @@ def run_custom_validators(sender, instance, **kwargs):
validator(instance) validator(instance)
@receiver(post_clean)
def run_save_validators(sender, instance, **kwargs):
model_name = f'{sender._meta.app_label}.{sender._meta.model_name}'
validators = get_config().CUSTOM_VALIDATORS.get(model_name, [])
run_validators(instance, validators)
@receiver(pre_delete)
def run_delete_validators(sender, instance, **kwargs):
model_name = f'{sender._meta.app_label}.{sender._meta.model_name}'
validators = get_config().PROTECTION_RULES.get(model_name, [])
try:
run_validators(instance, validators)
except ValidationError as e:
raise AbortRequest(
_("Deletion is prevented by a protection rule: {message}").format(
message=e
)
)
# #
# Dynamic configuration # Dynamic configuration
# #

View File

@ -1,10 +1,13 @@
from django.conf import settings from django.conf import settings
from django.core.exceptions import ValidationError from django.core.exceptions import ValidationError
from django.db import transaction
from django.test import TestCase, override_settings from django.test import TestCase, override_settings
from ipam.models import ASN, RIR from ipam.models import ASN, RIR
from dcim.choices import SiteStatusChoices
from dcim.models import Site from dcim.models import Site
from extras.validators import CustomValidator from extras.validators import CustomValidator
from utilities.exceptions import AbortRequest
class MyValidator(CustomValidator): class MyValidator(CustomValidator):
@ -14,6 +17,20 @@ class MyValidator(CustomValidator):
self.fail("Name must be foo!") self.fail("Name must be foo!")
eq_validator = CustomValidator({
'asn': {
'eq': 100
}
})
neq_validator = CustomValidator({
'asn': {
'neq': 100
}
})
min_validator = CustomValidator({ min_validator = CustomValidator({
'asn': { 'asn': {
'min': 65000 'min': 65000
@ -77,6 +94,18 @@ class CustomValidatorTest(TestCase):
validator = settings.CUSTOM_VALIDATORS['ipam.asn'][0] validator = settings.CUSTOM_VALIDATORS['ipam.asn'][0]
self.assertIsInstance(validator, CustomValidator) self.assertIsInstance(validator, CustomValidator)
@override_settings(CUSTOM_VALIDATORS={'ipam.asn': [eq_validator]})
def test_eq(self):
ASN(asn=100, rir=RIR.objects.first()).clean()
with self.assertRaises(ValidationError):
ASN(asn=99, rir=RIR.objects.first()).clean()
@override_settings(CUSTOM_VALIDATORS={'ipam.asn': [neq_validator]})
def test_neq(self):
ASN(asn=99, rir=RIR.objects.first()).clean()
with self.assertRaises(ValidationError):
ASN(asn=100, rir=RIR.objects.first()).clean()
@override_settings(CUSTOM_VALIDATORS={'ipam.asn': [min_validator]}) @override_settings(CUSTOM_VALIDATORS={'ipam.asn': [min_validator]})
def test_min(self): def test_min(self):
with self.assertRaises(ValidationError): with self.assertRaises(ValidationError):
@ -147,7 +176,7 @@ class CustomValidatorConfigTest(TestCase):
@override_settings( @override_settings(
CUSTOM_VALIDATORS={ CUSTOM_VALIDATORS={
'dcim.site': ( 'dcim.site': (
'extras.tests.test_customvalidator.MyValidator', 'extras.tests.test_customvalidation.MyValidator',
) )
} }
) )
@ -159,3 +188,62 @@ class CustomValidatorConfigTest(TestCase):
Site(name='foo', slug='foo').clean() Site(name='foo', slug='foo').clean()
with self.assertRaises(ValidationError): with self.assertRaises(ValidationError):
Site(name='bar', slug='bar').clean() Site(name='bar', slug='bar').clean()
class ProtectionRulesConfigTest(TestCase):
@override_settings(
PROTECTION_RULES={
'dcim.site': [
{'status': {'eq': SiteStatusChoices.STATUS_DECOMMISSIONING}}
]
}
)
def test_plain_data(self):
"""
Test custom validator configuration using plain data (as opposed to a CustomValidator
class)
"""
# Create a site with a protected status
site = Site(name='Site 1', slug='site-1', status=SiteStatusChoices.STATUS_ACTIVE)
site.save()
# Try to delete it
with self.assertRaises(AbortRequest):
with transaction.atomic():
site.delete()
# Change its status to an allowed value
site.status = SiteStatusChoices.STATUS_DECOMMISSIONING
site.save()
# Deletion should now succeed
site.delete()
@override_settings(
PROTECTION_RULES={
'dcim.site': (
'extras.tests.test_customvalidation.MyValidator',
)
}
)
def test_dotted_path(self):
"""
Test custom validator configuration using a dotted path (string) reference to a
CustomValidator class.
"""
# Create a site with a protected name
site = Site(name='bar', slug='bar')
site.save()
# Try to delete it
with self.assertRaises(AbortRequest):
with transaction.atomic():
site.delete()
# Change the name to an allowed value
site.name = site.slug = 'foo'
site.save()
# Deletion should now succeed
site.delete()

View File

@ -1,15 +1,38 @@
from django.core.exceptions import ValidationError
from django.core import validators from django.core import validators
from django.core.exceptions import ValidationError
from django.utils.translation import gettext_lazy as _
# NOTE: As this module may be imported by configuration.py, we cannot import # NOTE: As this module may be imported by configuration.py, we cannot import
# anything from NetBox itself. # anything from NetBox itself.
class IsEqualValidator(validators.BaseValidator):
"""
Employed by CustomValidator to require a specific value.
"""
message = _("Ensure this value is equal to %(limit_value)s.")
code = "is_equal"
def compare(self, a, b):
return a != b
class IsNotEqualValidator(validators.BaseValidator):
"""
Employed by CustomValidator to exclude a specific value.
"""
message = _("Ensure this value does not equal %(limit_value)s.")
code = "is_not_equal"
def compare(self, a, b):
return a == b
class IsEmptyValidator: class IsEmptyValidator:
""" """
Employed by CustomValidator to enforce required fields. Employed by CustomValidator to enforce required fields.
""" """
message = "This field must be empty." message = _("This field must be empty.")
code = 'is_empty' code = 'is_empty'
def __init__(self, enforce=True): def __init__(self, enforce=True):
@ -24,7 +47,7 @@ class IsNotEmptyValidator:
""" """
Employed by CustomValidator to enforce prohibited fields. Employed by CustomValidator to enforce prohibited fields.
""" """
message = "This field must not be empty." message = _("This field must not be empty.")
code = 'not_empty' code = 'not_empty'
def __init__(self, enforce=True): def __init__(self, enforce=True):
@ -50,6 +73,8 @@ class CustomValidator:
:param validation_rules: A dictionary mapping object attributes to validation rules :param validation_rules: A dictionary mapping object attributes to validation rules
""" """
VALIDATORS = { VALIDATORS = {
'eq': IsEqualValidator,
'neq': IsNotEqualValidator,
'min': validators.MinValueValidator, 'min': validators.MinValueValidator,
'max': validators.MaxValueValidator, 'max': validators.MaxValueValidator,
'min_length': validators.MinLengthValidator, 'min_length': validators.MinLengthValidator,

View File

@ -372,14 +372,14 @@ class IPAddressForm(TenancyForm, NetBoxModelForm):
# Do not allow assigning a network ID or broadcast address to an interface. # Do not allow assigning a network ID or broadcast address to an interface.
if interface and (address := self.cleaned_data.get('address')): if interface and (address := self.cleaned_data.get('address')):
if address.ip == address.network: if address.ip == address.network:
msg = _("{address} is a network ID, which may not be assigned to an interface.").format(address=address) msg = _("{ip} is a network ID, which may not be assigned to an interface.").format(ip=address.ip)
if address.version == 4 and address.prefixlen not in (31, 32): if address.version == 4 and address.prefixlen not in (31, 32):
raise ValidationError(msg) raise ValidationError(msg)
if address.version == 6 and address.prefixlen not in (127, 128): if address.version == 6 and address.prefixlen not in (127, 128):
raise ValidationError(msg) raise ValidationError(msg)
if address.version == 4 and address.ip == address.broadcast and address.prefixlen not in (31, 32): if address.version == 4 and address.ip == address.broadcast and address.prefixlen not in (31, 32):
msg = _("{address} is a broadcast address, which may not be assigned to an interface.").format( msg = _("{ip} is a broadcast address, which may not be assigned to an interface.").format(
address=address ip=address.ip
) )
raise ValidationError(msg) raise ValidationError(msg)

View File

@ -140,8 +140,11 @@ class Aggregate(GetAvailablePrefixesMixin, PrimaryModel):
if covering_aggregates: if covering_aggregates:
raise ValidationError({ raise ValidationError({
'prefix': _( 'prefix': _(
"Aggregates cannot overlap. {} is already covered by an existing aggregate ({})." "Aggregates cannot overlap. {prefix} is already covered by an existing aggregate ({aggregate})."
).format(self.prefix, covering_aggregates[0]) ).format(
prefix=self.prefix,
aggregate=covering_aggregates[0]
)
}) })
# Ensure that the aggregate being added does not cover an existing aggregate # Ensure that the aggregate being added does not cover an existing aggregate
@ -150,8 +153,11 @@ class Aggregate(GetAvailablePrefixesMixin, PrimaryModel):
covered_aggregates = covered_aggregates.exclude(pk=self.pk) covered_aggregates = covered_aggregates.exclude(pk=self.pk)
if covered_aggregates: if covered_aggregates:
raise ValidationError({ raise ValidationError({
'prefix': _("Aggregates cannot overlap. {} covers an existing aggregate ({}).").format( 'prefix': _(
self.prefix, covered_aggregates[0] "Prefixes cannot overlap aggregates. {prefix} covers an existing aggregate ({aggregate})."
).format(
prefix=self.prefix,
aggregate=covered_aggregates[0]
) )
}) })
@ -314,10 +320,11 @@ class Prefix(GetAvailablePrefixesMixin, PrimaryModel):
if (self.vrf is None and get_config().ENFORCE_GLOBAL_UNIQUE) or (self.vrf and self.vrf.enforce_unique): if (self.vrf is None and get_config().ENFORCE_GLOBAL_UNIQUE) or (self.vrf and self.vrf.enforce_unique):
duplicate_prefixes = self.get_duplicates() duplicate_prefixes = self.get_duplicates()
if duplicate_prefixes: if duplicate_prefixes:
table = _("VRF {vrf}").format(vrf=self.vrf) if self.vrf else _("global table")
raise ValidationError({ raise ValidationError({
'prefix': _("Duplicate prefix found in {}: {}").format( 'prefix': _("Duplicate prefix found in {table}: {prefix}").format(
_("VRF {}").format(self.vrf) if self.vrf else _("global table"), table=table,
duplicate_prefixes.first(), prefix=duplicate_prefixes.first(),
) )
}) })
@ -843,10 +850,11 @@ class IPAddress(PrimaryModel):
self.role not in IPADDRESS_ROLES_NONUNIQUE or self.role not in IPADDRESS_ROLES_NONUNIQUE or
any(dip.role not in IPADDRESS_ROLES_NONUNIQUE for dip in duplicate_ips) any(dip.role not in IPADDRESS_ROLES_NONUNIQUE for dip in duplicate_ips)
): ):
table = _("VRF {vrf}").format(vrf=self.vrf) if self.vrf else _("global table")
raise ValidationError({ raise ValidationError({
'address': _("Duplicate IP address found in {}: {}").format( 'address': _("Duplicate IP address found in {table}: {ipaddress}").format(
_("VRF {}").format(self.vrf) if self.vrf else _("global table"), table=table,
duplicate_ips.first(), ipaddress=duplicate_ips.first(),
) )
}) })

View File

@ -234,8 +234,8 @@ class VLAN(PrimaryModel):
if self.group and not self.group.min_vid <= self.vid <= self.group.max_vid: if self.group and not self.group.min_vid <= self.vid <= self.group.max_vid:
raise ValidationError({ raise ValidationError({
'vid': _( 'vid': _(
"VID must be between {min_vid} and {max_vid} for VLANs in group {group}" "VID must be between {minimum} and {maximum} for VLANs in group {group}"
).format(min_vid=self.group.min_vid, max_vid=self.group.max_vid, group=self.group) ).format(minimum=self.group.min_vid, maximum=self.group.max_vid, group=self.group)
}) })
def get_status_color(self): def get_status_color(self):

View File

@ -2,7 +2,7 @@ import logging
from django.core.exceptions import ObjectDoesNotExist, PermissionDenied from django.core.exceptions import ObjectDoesNotExist, PermissionDenied
from django.db import transaction from django.db import transaction
from django.db.models import ProtectedError from django.db.models import ProtectedError, RestrictedError
from django_pglocks import advisory_lock from django_pglocks import advisory_lock
from netbox.constants import ADVISORY_LOCK_KEYS from netbox.constants import ADVISORY_LOCK_KEYS
from rest_framework import mixins as drf_mixins from rest_framework import mixins as drf_mixins
@ -91,8 +91,11 @@ class NetBoxModelViewSet(
try: try:
return super().dispatch(request, *args, **kwargs) return super().dispatch(request, *args, **kwargs)
except ProtectedError as e: except (ProtectedError, RestrictedError) as e:
if type(e) is ProtectedError:
protected_objects = list(e.protected_objects) protected_objects = list(e.protected_objects)
else:
protected_objects = list(e.restricted_objects)
msg = f'Unable to delete object. {len(protected_objects)} dependent objects were found: ' msg = f'Unable to delete object. {len(protected_objects)} dependent objects were found: '
msg += ', '.join([f'{obj} ({obj.pk})' for obj in protected_objects]) msg += ', '.join([f'{obj} ({obj.pk})' for obj in protected_objects])
logger.warning(msg) logger.warning(msg)

View File

@ -137,11 +137,14 @@ class BulkUpdateModelMixin:
} }
] ]
""" """
def get_bulk_update_queryset(self):
return self.get_queryset()
def bulk_update(self, request, *args, **kwargs): def bulk_update(self, request, *args, **kwargs):
partial = kwargs.pop('partial', False) partial = kwargs.pop('partial', False)
serializer = BulkOperationSerializer(data=request.data, many=True) serializer = BulkOperationSerializer(data=request.data, many=True)
serializer.is_valid(raise_exception=True) serializer.is_valid(raise_exception=True)
qs = self.get_queryset().filter( qs = self.get_bulk_update_queryset().filter(
pk__in=[o['id'] for o in serializer.data] pk__in=[o['id'] for o in serializer.data]
) )
@ -184,10 +187,13 @@ class BulkDestroyModelMixin:
{"id": 456} {"id": 456}
] ]
""" """
def get_bulk_destroy_queryset(self):
return self.get_queryset()
def bulk_destroy(self, request, *args, **kwargs): def bulk_destroy(self, request, *args, **kwargs):
serializer = BulkOperationSerializer(data=request.data, many=True) serializer = BulkOperationSerializer(data=request.data, many=True)
serializer.is_valid(raise_exception=True) serializer.is_valid(raise_exception=True)
qs = self.get_queryset().filter( qs = self.get_bulk_destroy_queryset().filter(
pk__in=[o['id'] for o in serializer.data] pk__in=[o['id'] for o in serializer.data]
) )

View File

@ -152,9 +152,17 @@ PARAMS = (
description=_("Custom validation rules (JSON)"), description=_("Custom validation rules (JSON)"),
field=forms.JSONField, field=forms.JSONField,
field_kwargs={ field_kwargs={
'widget': forms.Textarea( 'widget': forms.Textarea(),
attrs={'class': 'vLargeTextField'} },
), ),
ConfigParam(
name='PROTECTION_RULES',
label=_('Protection rules'),
default={},
description=_("Deletion protection rules (JSON)"),
field=forms.JSONField,
field_kwargs={
'widget': forms.Textarea(),
}, },
), ),

View File

@ -0,0 +1,53 @@
from contextlib import contextmanager
from urllib.parse import urlparse
__all__ = (
'DataBackend',
)
class DataBackend:
"""
A data backend represents a specific system of record for data, such as a git repository or Amazon S3 bucket.
Attributes:
name: The identifier under which this backend will be registered in NetBox
label: The human-friendly name for this backend
is_local: A boolean indicating whether this backend accesses local data
parameters: A dictionary mapping configuration form field names to their classes
sensitive_parameters: An iterable of field names for which the values should not be displayed to the user
"""
is_local = False
parameters = {}
sensitive_parameters = []
# Prevent Django's template engine from calling the backend
# class when referenced via DataSource.backend_class
do_not_call_in_templates = True
def __init__(self, url, **kwargs):
self.url = url
self.params = kwargs
self.config = self.init_config()
def init_config(self):
"""
A hook to initialize the instance's configuration. The data returned by this method is assigned to the
instance's `config` attribute upon initialization, which can be referenced by the `fetch()` method.
"""
return
@property
def url_scheme(self):
return urlparse(self.url).scheme.lower()
@contextmanager
def fetch(self):
"""
A context manager which performs the following:
1. Handles all setup and synchronization
2. Yields the local path at which data has been replicated
3. Performs any necessary cleanup
"""
raise NotImplemented()

View File

@ -1,4 +1,4 @@
from django.utils.translation import gettext as _ from django.utils.translation import gettext_lazy as _
from netbox.registry import registry from netbox.registry import registry
from utilities.choices import ButtonColorChoices from utilities.choices import ButtonColorChoices

View File

@ -8,6 +8,7 @@ from packaging import version
from netbox.registry import registry from netbox.registry import registry
from netbox.search import register_search from netbox.search import register_search
from netbox.utils import register_data_backend
from .navigation import * from .navigation import *
from .registration import * from .registration import *
from .templates import * from .templates import *
@ -24,6 +25,7 @@ registry['plugins'].update({
DEFAULT_RESOURCE_PATHS = { DEFAULT_RESOURCE_PATHS = {
'search_indexes': 'search.indexes', 'search_indexes': 'search.indexes',
'data_backends': 'data_backends.backends',
'graphql_schema': 'graphql.schema', 'graphql_schema': 'graphql.schema',
'menu': 'navigation.menu', 'menu': 'navigation.menu',
'menu_items': 'navigation.menu_items', 'menu_items': 'navigation.menu_items',
@ -70,6 +72,7 @@ class PluginConfig(AppConfig):
# Optional plugin resources # Optional plugin resources
search_indexes = None search_indexes = None
data_backends = None
graphql_schema = None graphql_schema = None
menu = None menu = None
menu_items = None menu_items = None
@ -98,6 +101,11 @@ class PluginConfig(AppConfig):
for idx in search_indexes: for idx in search_indexes:
register_search(idx) register_search(idx)
# Register data backends (if defined)
data_backends = self._load_resource('data_backends') or []
for backend in data_backends:
register_data_backend()(backend)
# Register template content (if defined) # Register template content (if defined)
if template_extensions := self._load_resource('template_extensions'): if template_extensions := self._load_resource('template_extensions'):
register_template_extensions(template_extensions) register_template_extensions(template_extensions)

View File

@ -0,0 +1,18 @@
from contextlib import contextmanager
from netbox.data_backends import DataBackend
class DummyBackend(DataBackend):
name = 'dummy'
label = 'Dummy'
is_local = True
@contextmanager
def fetch(self):
yield '/tmp'
backends = (
DummyBackend,
)

View File

@ -6,6 +6,7 @@ from django.test import Client, TestCase, override_settings
from django.urls import reverse from django.urls import reverse
from netbox.tests.dummy_plugin import config as dummy_config from netbox.tests.dummy_plugin import config as dummy_config
from netbox.tests.dummy_plugin.data_backends import DummyBackend
from netbox.plugins.navigation import PluginMenu from netbox.plugins.navigation import PluginMenu
from netbox.plugins.utils import get_plugin_config from netbox.plugins.utils import get_plugin_config
from netbox.graphql.schema import Query from netbox.graphql.schema import Query
@ -111,6 +112,13 @@ class PluginTest(TestCase):
""" """
self.assertIn('netbox.tests.dummy_plugin.middleware.DummyMiddleware', settings.MIDDLEWARE) self.assertIn('netbox.tests.dummy_plugin.middleware.DummyMiddleware', settings.MIDDLEWARE)
def test_data_backends(self):
"""
Check registered data backends.
"""
self.assertIn('dummy', registry['data_backends'])
self.assertIs(registry['data_backends']['dummy'], DummyBackend)
def test_queues(self): def test_queues(self):
""" """
Check that plugin queues are registered with the accurate name. Check that plugin queues are registered with the accurate name.

26
netbox/netbox/utils.py Normal file
View File

@ -0,0 +1,26 @@
from netbox.registry import registry
__all__ = (
'get_data_backend_choices',
'register_data_backend',
)
def get_data_backend_choices():
return [
(None, '---------'),
*[
(name, cls.label) for name, cls in registry['data_backends'].items()
]
]
def register_data_backend():
"""
Decorator for registering a DataBackend class.
"""
def _wrapper(cls):
registry['data_backends'][cls.name] = cls
return cls
return _wrapper

View File

@ -7,7 +7,7 @@ from django.contrib.contenttypes.fields import GenericRel
from django.contrib.contenttypes.models import ContentType from django.contrib.contenttypes.models import ContentType
from django.core.exceptions import FieldDoesNotExist, ObjectDoesNotExist, ValidationError from django.core.exceptions import FieldDoesNotExist, ObjectDoesNotExist, ValidationError
from django.db import transaction, IntegrityError from django.db import transaction, IntegrityError
from django.db.models import ManyToManyField, ProtectedError from django.db.models import ManyToManyField, ProtectedError, RestrictedError
from django.db.models.fields.reverse_related import ManyToManyRel from django.db.models.fields.reverse_related import ManyToManyRel
from django.forms import HiddenInput, ModelMultipleChoiceField, MultipleHiddenInput from django.forms import HiddenInput, ModelMultipleChoiceField, MultipleHiddenInput
from django.http import HttpResponse from django.http import HttpResponse
@ -798,14 +798,15 @@ class BulkDeleteView(GetReturnURLMixin, BaseMultiObjectView):
queryset = self.queryset.filter(pk__in=pk_list) queryset = self.queryset.filter(pk__in=pk_list)
deleted_count = queryset.count() deleted_count = queryset.count()
try: try:
with transaction.atomic():
for obj in queryset: for obj in queryset:
# Take a snapshot of change-logged models # Take a snapshot of change-logged models
if hasattr(obj, 'snapshot'): if hasattr(obj, 'snapshot'):
obj.snapshot() obj.snapshot()
obj.delete() obj.delete()
except ProtectedError as e: except (ProtectedError, RestrictedError) as e:
logger.info("Caught ProtectedError while attempting to delete objects") logger.info(f"Caught {type(e)} while attempting to delete objects")
handle_protectederror(queryset, request, e) handle_protectederror(queryset, request, e)
return redirect(self.get_return_url(request)) return redirect(self.get_return_url(request))

View File

@ -1,9 +1,11 @@
import logging import logging
from collections import defaultdict
from copy import deepcopy from copy import deepcopy
from django.contrib import messages from django.contrib import messages
from django.db import transaction from django.db import router, transaction
from django.db.models import ProtectedError from django.db.models import ProtectedError, RestrictedError
from django.db.models.deletion import Collector
from django.shortcuts import redirect, render from django.shortcuts import redirect, render
from django.urls import reverse from django.urls import reverse
from django.utils.html import escape from django.utils.html import escape
@ -320,6 +322,27 @@ class ObjectDeleteView(GetReturnURLMixin, BaseObjectView):
def get_required_permission(self): def get_required_permission(self):
return get_permission_for_model(self.queryset.model, 'delete') return get_permission_for_model(self.queryset.model, 'delete')
def _get_dependent_objects(self, obj):
"""
Returns a dictionary mapping of dependent objects (organized by model) which will be deleted as a result of
deleting the requested object.
Args:
obj: The object to return dependent objects for
"""
using = router.db_for_write(obj._meta.model)
collector = Collector(using=using)
collector.collect([obj])
# Compile a mapping of models to instances
dependent_objects = defaultdict(list)
for model, instance in collector.instances_with_model():
# Omit the root object
if instance != obj:
dependent_objects[model].append(instance)
return dict(dependent_objects)
# #
# Request handlers # Request handlers
# #
@ -333,6 +356,7 @@ class ObjectDeleteView(GetReturnURLMixin, BaseObjectView):
""" """
obj = self.get_object(**kwargs) obj = self.get_object(**kwargs)
form = ConfirmationForm(initial=request.GET) form = ConfirmationForm(initial=request.GET)
dependent_objects = self._get_dependent_objects(obj)
# If this is an HTMX request, return only the rendered deletion form as modal content # If this is an HTMX request, return only the rendered deletion form as modal content
if is_htmx(request): if is_htmx(request):
@ -343,6 +367,7 @@ class ObjectDeleteView(GetReturnURLMixin, BaseObjectView):
'object_type': self.queryset.model._meta.verbose_name, 'object_type': self.queryset.model._meta.verbose_name,
'form': form, 'form': form,
'form_url': form_url, 'form_url': form_url,
'dependent_objects': dependent_objects,
**self.get_extra_context(request, obj), **self.get_extra_context(request, obj),
}) })
@ -350,6 +375,7 @@ class ObjectDeleteView(GetReturnURLMixin, BaseObjectView):
'object': obj, 'object': obj,
'form': form, 'form': form,
'return_url': self.get_return_url(request, obj), 'return_url': self.get_return_url(request, obj),
'dependent_objects': dependent_objects,
**self.get_extra_context(request, obj), **self.get_extra_context(request, obj),
}) })
@ -374,8 +400,8 @@ class ObjectDeleteView(GetReturnURLMixin, BaseObjectView):
try: try:
obj.delete() obj.delete()
except ProtectedError as e: except (ProtectedError, RestrictedError) as e:
logger.info("Caught ProtectedError while attempting to delete object") logger.info(f"Caught {type(e)} while attempting to delete objects")
handle_protectederror([obj], request, e) handle_protectederror([obj], request, e)
return redirect(obj.get_absolute_url()) return redirect(obj.get_absolute_url())

View File

@ -29,6 +29,16 @@
<th scope="row">{% trans "Description" %}</th> <th scope="row">{% trans "Description" %}</th>
<td>{{ object.description|placeholder }}</td> <td>{{ object.description|placeholder }}</td>
</tr> </tr>
<tr>
<th scope="row">{% trans "Color" %}</th>
<td>
{% if object.color %}
<span class="badge color-label" style="background-color: #{{ object.color }}">&nbsp;</span>
{% else %}
{{ ''|placeholder }}
{% endif %}
</td>
</tr>
</table> </table>
</div> </div>
</div> </div>

View File

@ -58,7 +58,7 @@
<tr> <tr>
<th scope="row">{% trans "URL" %}</th> <th scope="row">{% trans "URL" %}</th>
<td> <td>
{% if not object.is_local %} {% if not object.type.is_local %}
<a href="{{ object.source_url }}">{{ object.source_url }}</a> <a href="{{ object.source_url }}">{{ object.source_url }}</a>
{% else %} {% else %}
{{ object.source_url }} {{ object.source_url }}

View File

@ -35,6 +35,12 @@
<th scope="row">{% trans "Status" %}</th> <th scope="row">{% trans "Status" %}</th>
<td>{% badge object.get_status_display object.get_status_color %}</td> <td>{% badge object.get_status_display object.get_status_color %}</td>
</tr> </tr>
{% if object.error %}
<tr>
<th scope="row">{% trans "Error" %}</th>
<td>{{ object.error }}</td>
</tr>
{% endif %}
<tr> <tr>
<th scope="row">{% trans "Created By" %}</th> <th scope="row">{% trans "Created By" %}</th>
<td>{{ object.user|placeholder }}</td> <td>{{ object.user|placeholder }}</td>

View File

@ -8,8 +8,8 @@
{% block message %} {% block message %}
<p> <p>
{% blocktrans trimmed %} {% blocktrans trimmed with device=devicebay.device %}
Are you sure you want to delete this device bay from <strong>{{ devicebay.device }}</strong>? Are you sure you want to delete this device bay from <strong>{{ device }}</strong>?
{% endblocktrans %} {% endblocktrans %}
</p> </p>
{% endblock %} {% endblock %}

View File

@ -40,6 +40,10 @@
<td>{% trans "Height (U" %})</td> <td>{% trans "Height (U" %})</td>
<td>{{ object.u_height|floatformat }}</td> <td>{{ object.u_height|floatformat }}</td>
</tr> </tr>
<tr>
<td>{% trans "Exclude From Utilization" %})</td>
<td>{% checkmark object.exclude_from_utilization %}</td>
</tr>
<tr> <tr>
<td>{% trans "Full Depth" %}</td> <td>{% trans "Full Depth" %}</td>
<td>{% checkmark object.is_full_depth %}</td> <td>{% checkmark object.is_full_depth %}</td>

View File

@ -151,6 +151,10 @@
<th scope="row">{% trans "Custom validators" %}</th> <th scope="row">{% trans "Custom validators" %}</th>
<td>{{ object.data.CUSTOM_VALIDATORS|placeholder }}</td> <td>{{ object.data.CUSTOM_VALIDATORS|placeholder }}</td>
</tr> </tr>
<tr>
<th scope="row">{% trans "Protection rules" %}</th>
<td>{{ object.data.PROTECTION_RULES|placeholder }}</td>
</tr>
</table> </table>
</div> </div>
</div> </div>

View File

@ -12,6 +12,40 @@
Are you sure you want to <strong class="text-danger">delete</strong> {{ object_type }} <strong>{{ object }}</strong>? Are you sure you want to <strong class="text-danger">delete</strong> {{ object_type }} <strong>{{ object }}</strong>?
{% endblocktrans %} {% endblocktrans %}
</p> </p>
{% if dependent_objects %}
<p>
{% trans "The following objects will be deleted as a result of this action." %}
</p>
<div class="accordion" id="deleteAccordion">
{% for model, instances in dependent_objects.items %}
<div class="accordion-item">
<h2 class="accordion-header" id="deleteheading{{ forloop.counter }}">
<button class="accordion-button collapsed" type="button" data-bs-toggle="collapse" data-bs-target="#collapse{{ forloop.counter }}" aria-expanded="false" aria-controls="collapse{{ forloop.counter }}">
{% with object_count=instances|length %}
{{ object_count }}
{% if object_count == 1 %}
{{ model|meta:"verbose_name" }}
{% else %}
{{ model|meta:"verbose_name_plural" }}
{% endif %}
{% endwith %}
</button>
</h2>
<div id="collapse{{ forloop.counter }}" class="accordion-collapse collapse" aria-labelledby="deleteheading{{ forloop.counter }}" data-bs-parent="#deleteAccordion">
<div class="accordion-body p-0">
<div class="list-group list-group-flush">
{% for instance in instances %}
{% with url=instance.get_absolute_url %}
<a {% if url %}href="{{ url }}" {% endif %}class="list-group-item list-group-item-action">{{ instance }}</a>
{% endwith %}
{% endfor %}
</div>
</div>
</div>
</div>
{% endfor %}
</div>
{% endif %}
{% render_form form %} {% render_form form %}
</div> </div>
<div class="modal-footer"> <div class="modal-footer">

File diff suppressed because it is too large Load Diff

View File

@ -386,5 +386,5 @@ class ObjectPermissionForm(BootstrapMixin, forms.ModelForm):
model.objects.filter(qs_filter_from_constraints(constraints, tokens)).exists() model.objects.filter(qs_filter_from_constraints(constraints, tokens)).exists()
except FieldError as e: except FieldError as e:
raise forms.ValidationError({ raise forms.ValidationError({
'constraints': _('Invalid filter for {model}: {e}').format(model=model, e=e) 'constraints': _('Invalid filter for {model}: {error}').format(model=model, error=e)
}) })

View File

@ -169,7 +169,7 @@ class UserConfig(models.Model):
elif key in d: elif key in d:
err_path = '.'.join(path.split('.')[:i + 1]) err_path = '.'.join(path.split('.')[:i + 1])
raise TypeError( raise TypeError(
_("Key '{err_path}' is a leaf node; cannot assign new keys").format(err_path=err_path) _("Key '{path}' is a leaf node; cannot assign new keys").format(path=err_path)
) )
else: else:
d = d.setdefault(key, {}) d = d.setdefault(key, {})

View File

@ -1,16 +1,26 @@
from django.contrib import messages from django.contrib import messages
from django.db.models import ProtectedError, RestrictedError
from django.utils.html import escape from django.utils.html import escape
from django.utils.safestring import mark_safe from django.utils.safestring import mark_safe
from django.utils.translation import gettext_lazy as _
def handle_protectederror(obj_list, request, e): def handle_protectederror(obj_list, request, e):
""" """
Generate a user-friendly error message in response to a ProtectedError exception. Generate a user-friendly error message in response to a ProtectedError or RestrictedError exception.
""" """
if type(e) is ProtectedError:
protected_objects = list(e.protected_objects) protected_objects = list(e.protected_objects)
protected_count = len(protected_objects) if len(protected_objects) <= 50 else 'More than 50' elif type(e) is RestrictedError:
err_message = f"Unable to delete <strong>{', '.join(str(obj) for obj in obj_list)}</strong>. " \ protected_objects = list(e.restricted_objects)
f"{protected_count} dependent objects were found: " else:
raise e
# Formulate the error message
err_message = _("Unable to delete <strong>{objects}</strong>. {count} dependent objects were found: ").format(
objects=', '.join(str(obj) for obj in obj_list),
count=len(protected_objects) if len(protected_objects) <= 50 else _('More than 50')
)
# Append dependent objects to error message # Append dependent objects to error message
dependent_objects = [] dependent_objects = []

View File

@ -4,6 +4,7 @@ from dcim.models import Device
from django.db.models import Sum from django.db.models import Sum
from extras.api.mixins import ConfigContextQuerySetMixin from extras.api.mixins import ConfigContextQuerySetMixin
from netbox.api.viewsets import NetBoxModelViewSet from netbox.api.viewsets import NetBoxModelViewSet
from utilities.query_functions import CollateAsChar
from utilities.utils import count_related from utilities.utils import count_related
from virtualization import filtersets from virtualization import filtersets
from virtualization.models import * from virtualization.models import *
@ -89,6 +90,10 @@ class VMInterfaceViewSet(NetBoxModelViewSet):
filterset_class = filtersets.VMInterfaceFilterSet filterset_class = filtersets.VMInterfaceFilterSet
brief_prefetch_fields = ['virtual_machine'] brief_prefetch_fields = ['virtual_machine']
def get_bulk_destroy_queryset(self):
# Ensure child interfaces are deleted prior to their parents
return self.get_queryset().order_by('virtual_machine', 'parent', CollateAsChar('_name'))
class VirtualDiskViewSet(NetBoxModelViewSet): class VirtualDiskViewSet(NetBoxModelViewSet):
queryset = VirtualDisk.objects.prefetch_related( queryset = VirtualDisk.objects.prefetch_related(

View File

@ -152,8 +152,12 @@ class ClusterAddDevicesForm(BootstrapMixin, forms.Form):
for device in self.cleaned_data.get('devices', []): for device in self.cleaned_data.get('devices', []):
if device.site != self.cluster.site: if device.site != self.cluster.site:
raise ValidationError({ raise ValidationError({
'devices': _("{} belongs to a different site ({}) than the cluster ({})").format( 'devices': _(
device, device.site, self.cluster.site "{device} belongs to a different site ({device_site}) than the cluster ({cluster_site})"
).format(
device=device,
device_site=device.site,
cluster_site=self.cluster.site
) )
}) })

View File

@ -0,0 +1,19 @@
# Generated by Django 4.2.6 on 2023-10-20 11:48
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
dependencies = [
('virtualization', '0036_virtualmachine_config_template'),
]
operations = [
migrations.AlterField(
model_name='vminterface',
name='parent',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.RESTRICT, related_name='child_interfaces', to='virtualization.vminterface'),
),
]

View File

@ -13,7 +13,7 @@ import utilities.tracking
class Migration(migrations.Migration): class Migration(migrations.Migration):
dependencies = [ dependencies = [
('extras', '0098_webhook_custom_field_data_webhook_tags'), ('extras', '0098_webhook_custom_field_data_webhook_tags'),
('virtualization', '0036_virtualmachine_config_template'), ('virtualization', '0037_protect_child_interfaces'),
] ]
operations = [ operations = [

View File

@ -135,10 +135,9 @@ class Cluster(ContactsMixin, PrimaryModel):
# If the Cluster is assigned to a Site, verify that all host Devices belong to that Site. # If the Cluster is assigned to a Site, verify that all host Devices belong to that Site.
if self.pk and self.site: if self.pk and self.site:
nonsite_devices = Device.objects.filter(cluster=self).exclude(site=self.site).count() if nonsite_devices := Device.objects.filter(cluster=self).exclude(site=self.site).count():
if nonsite_devices:
raise ValidationError({ raise ValidationError({
'site': _("{} devices are assigned as hosts for this cluster but are not in site {}").format( 'site': _(
nonsite_devices, self.site "{count} devices are assigned as hosts for this cluster but are not in site {site}"
) ).format(count=nonsite_devices, site=self.site)
}) })

View File

@ -294,6 +294,32 @@ class VMInterfaceTest(APIViewTestCases.APIViewTestCase):
}, },
] ]
def test_bulk_delete_child_interfaces(self):
interface1 = VMInterface.objects.get(name='Interface 1')
virtual_machine = interface1.virtual_machine
self.add_permissions('virtualization.delete_vminterface')
# Create a child interface
child = VMInterface.objects.create(
virtual_machine=virtual_machine,
name='Interface 1A',
parent=interface1
)
self.assertEqual(virtual_machine.interfaces.count(), 4)
# Attempt to delete only the parent interface
url = self._get_detail_url(interface1)
self.client.delete(url, **self.header)
self.assertEqual(virtual_machine.interfaces.count(), 4) # Parent was not deleted
# Attempt to bulk delete parent & child together
data = [
{"id": interface1.pk},
{"id": child.pk},
]
self.client.delete(self._get_list_url(), data, format='json', **self.header)
self.assertEqual(virtual_machine.interfaces.count(), 2) # Child & parent were both deleted
class VirtualDiskTest(APIViewTestCases.APIViewTestCase): class VirtualDiskTest(APIViewTestCases.APIViewTestCase):
model = VirtualDisk model = VirtualDisk

View File

@ -375,6 +375,35 @@ class VMInterfaceTestCase(ViewTestCases.DeviceComponentViewTestCase):
'tagged_vlans': [v.pk for v in vlans[1:4]], 'tagged_vlans': [v.pk for v in vlans[1:4]],
} }
def test_bulk_delete_child_interfaces(self):
interface1 = VMInterface.objects.get(name='Interface 1')
virtual_machine = interface1.virtual_machine
self.add_permissions('virtualization.delete_vminterface')
# Create a child interface
child = VMInterface.objects.create(
virtual_machine=virtual_machine,
name='Interface 1A',
parent=interface1
)
self.assertEqual(virtual_machine.interfaces.count(), 4)
# Attempt to delete only the parent interface
data = {
'confirm': True,
}
self.client.post(self._get_url('delete', interface1), data)
self.assertEqual(virtual_machine.interfaces.count(), 4) # Parent was not deleted
# Attempt to bulk delete parent & child together
data = {
'pk': [interface1.pk, child.pk],
'confirm': True,
'_confirm': True, # Form button
}
self.client.post(self._get_url('bulk_delete'), data)
self.assertEqual(virtual_machine.interfaces.count(), 2) # Child & parent were both deleted
class VirtualDiskTestCase(ViewTestCases.DeviceComponentViewTestCase): class VirtualDiskTestCase(ViewTestCases.DeviceComponentViewTestCase):
model = VirtualDisk model = VirtualDisk

View File

@ -1,5 +1,4 @@
import traceback import traceback
from collections import defaultdict
from django.contrib import messages from django.contrib import messages
from django.db import transaction from django.db import transaction
@ -19,6 +18,7 @@ from ipam.tables import InterfaceVLANTable
from netbox.constants import DEFAULT_ACTION_PERMISSIONS from netbox.constants import DEFAULT_ACTION_PERMISSIONS
from netbox.views import generic from netbox.views import generic
from tenancy.views import ObjectContactsView from tenancy.views import ObjectContactsView
from utilities.query_functions import CollateAsChar
from utilities.utils import count_related from utilities.utils import count_related
from utilities.views import ViewTab, register_model_view from utilities.views import ViewTab, register_model_view
from . import filtersets, forms, tables from . import filtersets, forms, tables
@ -575,7 +575,8 @@ class VMInterfaceBulkRenameView(generic.BulkRenameView):
class VMInterfaceBulkDeleteView(generic.BulkDeleteView): class VMInterfaceBulkDeleteView(generic.BulkDeleteView):
queryset = VMInterface.objects.all() # Ensure child interfaces are deleted prior to their parents
queryset = VMInterface.objects.order_by('virtual_machine', 'parent', CollateAsChar('_name'))
filterset = filtersets.VMInterfaceFilterSet filterset = filtersets.VMInterfaceFilterSet
table = tables.VMInterfaceTable table = tables.VMInterfaceTable

View File

@ -213,14 +213,14 @@ class WirelessLink(WirelessAuthenticationBase, PrimaryModel):
if self.interface_a.type not in WIRELESS_IFACE_TYPES: if self.interface_a.type not in WIRELESS_IFACE_TYPES:
raise ValidationError({ raise ValidationError({
'interface_a': _( 'interface_a': _(
"{type_display} is not a wireless interface." "{type} is not a wireless interface."
).format(type_display=self.interface_a.get_type_display()) ).format(type=self.interface_a.get_type_display())
}) })
if self.interface_b.type not in WIRELESS_IFACE_TYPES: if self.interface_b.type not in WIRELESS_IFACE_TYPES:
raise ValidationError({ raise ValidationError({
'interface_a': _( 'interface_a': _(
"{type_display} is not a wireless interface." "{type} is not a wireless interface."
).format(type_display=self.interface_b.get_type_display()) ).format(type=self.interface_b.get_type_display())
}) })
def save(self, *args, **kwargs): def save(self, *args, **kwargs):