Merge pull request #5865 from netbox-community/develop

Release v2.10.5

.github/stale.yml

@@ -1,8 +1,5 @@
 # Configuration for Stale (https://github.com/apps/stale)
 
-# Pull requests are exempt from being marked as stale
-only: issues
-
 # Number of days of inactivity before an issue becomes stale
 daysUntilStale: 45
 

docs/additional-features/prometheus-metrics.md

@@ -26,4 +26,4 @@ For the exhaustive list of exposed metrics, visit the `/metrics` endpoint on you
 When deploying NetBox in a multiprocess manner (e.g. running multiple Gunicorn workers) the Prometheus client library requires the use of a shared directory to collect metrics from all worker processes. To configure this, first create or designate a local directory to which the worker processes have read and write access, and then configure your WSGI service (e.g. Gunicorn) to define this path as the `prometheus_multiproc_dir` environment variable.
 
 !!! warning
-    If having accurate long-term metrics in a multiprocess environment is crucial to your deployment, it's recommended you use the `uwsgi` library instead of `gunicorn`. The issue lies in the way `gunicorn` tracks worker processes (vs `uwsgi`) which helps manage the metrics files created by the above configurations. If you're using Netbox with gunicorn in a containerized enviroment following the one-process-per-container methodology, then you will likely not need to change to `uwsgi`. More details can be found in  [issue #3779](https://github.com/netbox-community/netbox/issues/3779#issuecomment-590547562).
+    If having accurate long-term metrics in a multiprocess environment is crucial to your deployment, it's recommended you use the `uwsgi` library instead of `gunicorn`. The issue lies in the way `gunicorn` tracks worker processes (vs `uwsgi`) which helps manage the metrics files created by the above configurations. If you're using NetBox with gunicorn in a containerized enviroment following the one-process-per-container methodology, then you will likely not need to change to `uwsgi`. More details can be found in  [issue #3779](https://github.com/netbox-community/netbox/issues/3779#issuecomment-590547562).

docs/additional-features/reports.md

@@ -66,7 +66,7 @@ class DeviceConnectionsReport(Report):
             for power_port in PowerPort.objects.filter(device=device):
                 if power_port.connected_endpoint is not None:
                     connected_ports += 1
-                    if not power_port.connection_status:
+                    if not power_port.path.is_active:
                         self.log_warning(
                             device,
                             "Power connection for {} marked as planned".format(power_port.name)

docs/installation/1-postgresql.md

@@ -51,7 +51,7 @@ $ sudo -u postgres psql
 psql (12.5 (Ubuntu 12.5-0ubuntu0.20.04.1))
 Type "help" for help.
 
-postgres=# CREATE DATABASE netbox;
+postgres=# CREATE DATABASE netbox ENCODING 'UTF8' LC_COLLATE='C.UTF-8' LC_CTYPE='C.UTF-8';
 CREATE DATABASE
 postgres=# CREATE USER netbox WITH PASSWORD 'J5brHrAXFLQSif0K';
 CREATE ROLE

docs/installation/6-ldap.md

@@ -140,7 +140,7 @@ AUTH_LDAP_CACHE_TIMEOUT = 3600
 
 ## Troubleshooting LDAP
 
-`systemctl restart netbox` restarts the Netbox service, and initiates any changes made to `ldap_config.py`. If there are syntax errors present, the NetBox process will not spawn an instance, and errors should be logged to `/var/log/messages`.
+`systemctl restart netbox` restarts the NetBox service, and initiates any changes made to `ldap_config.py`. If there are syntax errors present, the NetBox process will not spawn an instance, and errors should be logged to `/var/log/messages`.
 
 For troubleshooting LDAP user/group queries, add or merge the following [logging](/configuration/optional-settings.md#logging) configuration to `configuration.py`:
 

docs/release-notes/version-2.10.md

@@ -1,5 +1,21 @@
 # NetBox v2.10
 
+## v2.10.5 (2021-02-24)
+
+### Bug Fixes
+
+* [#5315](https://github.com/netbox-community/netbox/issues/5315) - Fix site unassignment from VLAN when using "None" option
+* [#5626](https://github.com/netbox-community/netbox/issues/5626) - Fix REST API representation for circuit terminations connected to non-interface endpoints
+* [#5716](https://github.com/netbox-community/netbox/issues/5716) - Fix filtering rack reservations by custom field
+* [#5718](https://github.com/netbox-community/netbox/issues/5718) - Fix bulk editing of services when no port(s) are defined
+* [#5735](https://github.com/netbox-community/netbox/issues/5735) - Ensure consistent treatment of duplicate IP addresses
+* [#5738](https://github.com/netbox-community/netbox/issues/5738) - Fix redirect to device components view after disconnecting a cable
+* [#5753](https://github.com/netbox-community/netbox/issues/5753) - Fix Redis Sentinel password application for caching
+* [#5786](https://github.com/netbox-community/netbox/issues/5786) - Allow setting null tenant group on tenant via REST API
+* [#5841](https://github.com/netbox-community/netbox/issues/5841) - Disallow the creation of available prefixes/IP addresses in violation of assigned permission constraints
+
+---
+
 ## v2.10.4 (2021-01-26)
 
 ### Enhancements

netbox/circuits/api/serializers.py

@@ -40,14 +40,16 @@ class CircuitTypeSerializer(ValidatedModelSerializer):
         fields = ['id', 'url', 'name', 'slug', 'description', 'circuit_count']
 
 
-class CircuitCircuitTerminationSerializer(WritableNestedSerializer):
+class CircuitCircuitTerminationSerializer(WritableNestedSerializer, ConnectedEndpointSerializer):
     url = serializers.HyperlinkedIdentityField(view_name='circuits-api:circuittermination-detail')
     site = NestedSiteSerializer()
-    connected_endpoint = NestedInterfaceSerializer()
 
     class Meta:
         model = CircuitTermination
-        fields = ['id', 'url', 'site', 'connected_endpoint', 'port_speed', 'upstream_speed', 'xconnect_id']
+        fields = [
+            'id', 'url', 'site', 'port_speed', 'upstream_speed', 'xconnect_id', 'connected_endpoint',
+            'connected_endpoint_type', 'connected_endpoint_reachable',
+        ]
 
 
 class CircuitSerializer(TaggedObjectSerializer, CustomFieldModelSerializer):

netbox/dcim/filters.py

@@ -264,7 +264,7 @@ class RackFilterSet(BaseFilterSet, TenancyFilterSet, CustomFieldModelFilterSet,
         )
 
 
-class RackReservationFilterSet(BaseFilterSet, TenancyFilterSet):
+class RackReservationFilterSet(BaseFilterSet, TenancyFilterSet, CustomFieldModelFilterSet):
     q = django_filters.CharFilter(
         method='search',
         label='Search',

netbox/dcim/tables/template_code.py

@@ -95,6 +95,11 @@ CONSOLEPORT_BUTTONS = """
 {% if record.cable %}
     <a href="{% url 'dcim:consoleport_trace' pk=record.pk %}" class="btn btn-primary btn-xs" title="Trace"><i class="mdi mdi-transit-connection-variant"></i></a>
     {% include 'dcim/inc/cable_toggle_buttons.html' with cable=record.cable %}
+    {% if perms.dcim.delete_cable %}
+        <a href="{% url 'dcim:cable_delete' pk=record.cable.pk %}?return_url={% url 'dcim:device_consoleports' pk=object.pk %}" title="Remove cable" class="btn btn-danger btn-xs">
+            <i class="mdi mdi-ethernet-cable-off" aria-hidden="true"></i>
+        </a>
+    {% endif %}
 {% elif perms.dcim.add_cable %}
     <a href="#" class="btn btn-default btn-xs disabled"><i class="mdi mdi-transit-connection-variant" aria-hidden="true"></i></a>
     <a href="#" class="btn btn-default btn-xs disabled"><i class="mdi mdi-lan-connect" aria-hidden="true"></i></a>
@@ -115,6 +120,11 @@ CONSOLESERVERPORT_BUTTONS = """
 {% if record.cable %}
     <a href="{% url 'dcim:consoleserverport_trace' pk=record.pk %}" class="btn btn-primary btn-xs" title="Trace"><i class="mdi mdi-transit-connection-variant"></i></a>
     {% include 'dcim/inc/cable_toggle_buttons.html' with cable=record.cable %}
+    {% if perms.dcim.delete_cable %}
+        <a href="{% url 'dcim:cable_delete' pk=record.cable.pk %}?return_url={% url 'dcim:device_consoleserverports' pk=object.pk %}" title="Remove cable" class="btn btn-danger btn-xs">
+            <i class="mdi mdi-ethernet-cable-off" aria-hidden="true"></i>
+        </a>
+    {% endif %}
 {% elif perms.dcim.add_cable %}
     <a href="#" class="btn btn-default btn-xs disabled"><i class="mdi mdi-transit-connection-variant" aria-hidden="true"></i></a>
     <a href="#" class="btn btn-default btn-xs disabled"><i class="mdi mdi-lan-connect" aria-hidden="true"></i></a>
@@ -135,6 +145,11 @@ POWERPORT_BUTTONS = """
 {% if record.cable %}
     <a href="{% url 'dcim:powerport_trace' pk=record.pk %}" class="btn btn-primary btn-xs" title="Trace"><i class="mdi mdi-transit-connection-variant"></i></a>
     {% include 'dcim/inc/cable_toggle_buttons.html' with cable=record.cable %}
+    {% if perms.dcim.delete_cable %}
+        <a href="{% url 'dcim:cable_delete' pk=record.cable.pk %}?return_url={% url 'dcim:device_powerports' pk=object.pk %}" title="Remove cable" class="btn btn-danger btn-xs">
+            <i class="mdi mdi-ethernet-cable-off" aria-hidden="true"></i>
+        </a>
+    {% endif %}
 {% elif perms.dcim.add_cable %}
     <a href="#" class="btn btn-default btn-xs disabled"><i class="mdi mdi-transit-connection-variant" aria-hidden="true"></i></a>
     <a href="#" class="btn btn-default btn-xs disabled"><i class="mdi mdi-lan-connect" aria-hidden="true"></i></a>
@@ -154,6 +169,11 @@ POWEROUTLET_BUTTONS = """
 {% if record.cable %}
     <a href="{% url 'dcim:poweroutlet_trace' pk=record.pk %}" class="btn btn-primary btn-xs" title="Trace"><i class="mdi mdi-transit-connection-variant"></i></a>
     {% include 'dcim/inc/cable_toggle_buttons.html' with cable=record.cable %}
+    {% if perms.dcim.delete_cable %}
+        <a href="{% url 'dcim:cable_delete' pk=record.cable.pk %}?return_url={% url 'dcim:device_poweroutlets' pk=object.pk %}" title="Remove cable" class="btn btn-danger btn-xs">
+            <i class="mdi mdi-ethernet-cable-off" aria-hidden="true"></i>
+        </a>
+    {% endif %}
 {% elif perms.dcim.add_cable %}
     <a href="#" class="btn btn-default btn-xs disabled"><i class="mdi mdi-transit-connection-variant" aria-hidden="true"></i></a>
     <a href="#" class="btn btn-default btn-xs disabled"><i class="mdi mdi-lan-connect" aria-hidden="true"></i></a>
@@ -172,6 +192,11 @@ INTERFACE_BUTTONS = """
 {% if record.cable %}
     <a href="{% url 'dcim:interface_trace' pk=record.pk %}" class="btn btn-primary btn-xs" title="Trace"><i class="mdi mdi-transit-connection-variant"></i></a>
     {% include 'dcim/inc/cable_toggle_buttons.html' with cable=record.cable %}
+    {% if perms.dcim.delete_cable %}
+        <a href="{% url 'dcim:cable_delete' pk=record.cable.pk %}?return_url={% url 'dcim:device_interfaces' pk=object.pk %}" title="Remove cable" class="btn btn-danger btn-xs">
+            <i class="mdi mdi-ethernet-cable-off" aria-hidden="true"></i>
+        </a>
+    {% endif %}
 {% elif record.is_connectable and perms.dcim.add_cable %}
     <a href="#" class="btn btn-default btn-xs disabled"><i class="mdi mdi-transit-connection-variant" aria-hidden="true"></i></a>
     <a href="#" class="btn btn-default btn-xs disabled"><i class="mdi mdi-lan-connect" aria-hidden="true"></i></a>
@@ -193,6 +218,11 @@ FRONTPORT_BUTTONS = """
 {% if record.cable %}
     <a href="{% url 'dcim:frontport_trace' pk=record.pk %}" class="btn btn-primary btn-xs" title="Trace"><i class="mdi mdi-transit-connection-variant"></i></a>
     {% include 'dcim/inc/cable_toggle_buttons.html' with cable=record.cable %}
+    {% if perms.dcim.delete_cable %}
+        <a href="{% url 'dcim:cable_delete' pk=record.cable.pk %}?return_url={% url 'dcim:device_frontports' pk=object.pk %}" title="Remove cable" class="btn btn-danger btn-xs">
+            <i class="mdi mdi-ethernet-cable-off" aria-hidden="true"></i>
+        </a>
+    {% endif %}
 {% elif perms.dcim.add_cable %}
     <a href="#" class="btn btn-default btn-xs disabled"><i class="mdi mdi-transit-connection-variant" aria-hidden="true"></i></a>
     <a href="#" class="btn btn-default btn-xs disabled"><i class="mdi mdi-lan-connect" aria-hidden="true"></i></a>
@@ -216,6 +246,11 @@ REARPORT_BUTTONS = """
 {% if record.cable %}
     <a href="{% url 'dcim:rearport_trace' pk=record.pk %}" class="btn btn-primary btn-xs" title="Trace"><i class="mdi mdi-transit-connection-variant"></i></a>
     {% include 'dcim/inc/cable_toggle_buttons.html' with cable=record.cable %}
+    {% if perms.dcim.delete_cable %}
+        <a href="{% url 'dcim:cable_delete' pk=record.cable.pk %}?return_url={% url 'dcim:device_rearports' pk=object.pk %}" title="Remove cable" class="btn btn-danger btn-xs">
+            <i class="mdi mdi-ethernet-cable-off" aria-hidden="true"></i>
+        </a>
+    {% endif %}
 {% elif perms.dcim.add_cable %}
     <a href="#" class="btn btn-default btn-xs disabled"><i class="mdi mdi-transit-connection-variant" aria-hidden="true"></i></a>
     <a href="#" class="btn btn-default btn-xs disabled"><i class="mdi mdi-lan-connect" aria-hidden="true"></i></a>

netbox/ipam/api/views.py

@@ -1,4 +1,6 @@
 from django.conf import settings
+from django.core.exceptions import ObjectDoesNotExist, PermissionDenied
+from django.db import transaction
 from django.shortcuts import get_object_or_404
 from django_pglocks import advisory_lock
 from drf_yasg.utils import swagger_auto_schema
@@ -162,7 +164,12 @@ class PrefixViewSet(CustomFieldModelViewSet):
 
             # Create the new Prefix(es)
             if serializer.is_valid():
-                serializer.save()
+                try:
+                    with transaction.atomic():
+                        created = serializer.save()
+                        self._validate_objects(created)
+                except ObjectDoesNotExist:
+                    raise PermissionDenied()
                 return Response(serializer.data, status=status.HTTP_201_CREATED)
 
             return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)
@@ -225,7 +232,12 @@ class PrefixViewSet(CustomFieldModelViewSet):
 
             # Create the new IP address(es)
             if serializer.is_valid():
-                serializer.save()
+                try:
+                    with transaction.atomic():
+                        created = serializer.save()
+                        self._validate_objects(created)
+                except ObjectDoesNotExist:
+                    raise PermissionDenied()
                 return Response(serializer.data, status=status.HTTP_201_CREATED)
 
             return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)

netbox/ipam/models.py

@@ -734,13 +734,12 @@ class IPAddress(ChangeLoggedModel, CustomFieldModel):
                 })
 
             # Enforce unique IP space (if applicable)
-            if self.role not in IPADDRESS_ROLES_NONUNIQUE and ((
+            if (self.vrf is None and settings.ENFORCE_GLOBAL_UNIQUE) or (self.vrf and self.vrf.enforce_unique):
-                self.vrf is None and settings.ENFORCE_GLOBAL_UNIQUE
-            ) or (
-                self.vrf and self.vrf.enforce_unique
-            )):
                 duplicate_ips = self.get_duplicates()
-                if duplicate_ips:
+                if duplicate_ips and (
+                        self.role not in IPADDRESS_ROLES_NONUNIQUE or
+                        any(dip.role not in IPADDRESS_ROLES_NONUNIQUE for dip in duplicate_ips)
+                ):
                     raise ValidationError({
                         'address': "Duplicate IP address found in {}: {}".format(
                             "VRF {}".format(self.vrf) if self.vrf else "global table",

netbox/ipam/tests/test_models.py

@@ -259,6 +259,18 @@ class TestIPAddress(TestCase):
         duplicate_ip = IPAddress(vrf=vrf, address=netaddr.IPNetwork('192.0.2.1/24'))
         self.assertRaises(ValidationError, duplicate_ip.clean)
 
+    @override_settings(ENFORCE_GLOBAL_UNIQUE=True)
+    def test_duplicate_nonunique_nonrole_role(self):
+        IPAddress.objects.create(address=netaddr.IPNetwork('192.0.2.1/24'))
+        duplicate_ip = IPAddress(address=netaddr.IPNetwork('192.0.2.1/24'), role=IPAddressRoleChoices.ROLE_VIP)
+        self.assertRaises(ValidationError, duplicate_ip.clean)
+
+    @override_settings(ENFORCE_GLOBAL_UNIQUE=True)
+    def test_duplicate_nonunique_role_nonrole(self):
+        IPAddress.objects.create(address=netaddr.IPNetwork('192.0.2.1/24'), role=IPAddressRoleChoices.ROLE_VIP)
+        duplicate_ip = IPAddress(address=netaddr.IPNetwork('192.0.2.1/24'))
+        self.assertRaises(ValidationError, duplicate_ip.clean)
+
     @override_settings(ENFORCE_GLOBAL_UNIQUE=True)
     def test_duplicate_nonunique_role(self):
         IPAddress.objects.create(address=netaddr.IPNetwork('192.0.2.1/24'), role=IPAddressRoleChoices.ROLE_VIP)

netbox/netbox/settings.py

@@ -16,7 +16,7 @@ from django.core.validators import URLValidator
 # Environment setup
 #
 
-VERSION = '2.10.4'
+VERSION = '2.10.5'
 
 # Hostname
 HOSTNAME = platform.node()
@@ -391,6 +391,7 @@ if CACHING_REDIS_USING_SENTINEL:
         'locations': CACHING_REDIS_SENTINELS,
         'service_name': CACHING_REDIS_SENTINEL_SERVICE,
         'db': CACHING_REDIS_DATABASE,
+        'password': CACHING_REDIS_PASSWORD,
     }
 else:
     if CACHING_REDIS_SSL:

netbox/netbox/views/generic.py

@@ -792,7 +792,7 @@ class BulkEditView(GetReturnURLMixin, ObjectPermissionRequiredMixin, View):
                                     if form.cleaned_data[name]:
                                         getattr(obj, name).set(form.cleaned_data[name])
                                 # Normal fields
-                                elif form.cleaned_data[name] not in (None, ''):
+                                elif form.cleaned_data[name] not in (None, '', []):
                                     setattr(obj, name, form.cleaned_data[name])
 
                             # Update custom fields

netbox/project-static/css/base.css

@@ -177,6 +177,10 @@ nav ul.pagination {
     margin-top: 0;
     margin-bottom: 8px !important;
 }
+.pagination > li > a > .mdi::before {
+    top: 0;
+    font-size: 14px;
+}
 
 /* Devices */
 table.component-list td.subtable {

netbox/templates/dcim/inc/cable_toggle_buttons.html

@@ -9,8 +9,3 @@
         </a>
     {% endif %}
 {% endif %}
-{% if perms.dcim.delete_cable %}
-    <a href="{% url 'dcim:cable_delete' pk=cable.pk %}?return_url={{ object.get_absolute_url }}" title="Remove cable" class="btn btn-danger btn-xs">
-            <i class="mdi mdi-ethernet-cable-off" aria-hidden="true"></i>
-    </a>
-{% endif %}

netbox/tenancy/api/serializers.py

@@ -24,7 +24,7 @@ class TenantGroupSerializer(ValidatedModelSerializer):
 
 class TenantSerializer(TaggedObjectSerializer, CustomFieldModelSerializer):
     url = serializers.HyperlinkedIdentityField(view_name='tenancy-api:tenant-detail')
-    group = NestedTenantGroupSerializer(required=False)
+    group = NestedTenantGroupSerializer(required=False, allow_null=True)
     circuit_count = serializers.IntegerField(read_only=True)
     device_count = serializers.IntegerField(read_only=True)
     ipaddress_count = serializers.IntegerField(read_only=True)

netbox/tenancy/tests/test_api.py

@@ -56,6 +56,7 @@ class TenantTest(APIViewTestCases.APIViewTestCase):
     model = Tenant
     brief_fields = ['id', 'name', 'slug', 'url']
     bulk_update_data = {
+        'group': None,
         'description': 'New description',
     }
 

netbox/utilities/forms/fields.py

@@ -5,6 +5,7 @@ from io import StringIO
 
 import django_filters
 from django import forms
+from django.conf import settings
 from django.forms.fields import JSONField as _JSONField, InvalidJSONInput
 from django.core.exceptions import MultipleObjectsReturned, ObjectDoesNotExist
 from django.db.models import Count
@@ -355,7 +356,15 @@ class DynamicModelChoiceField(DynamicModelChoiceMixin, forms.ModelChoiceField):
     Override get_bound_field() to avoid pre-populating field choices with a SQL query. The field will be
     rendered only with choices set via bound data. Choices are populated on-demand via the APISelect widget.
     """
-    pass
+
+    def clean(self, value):
+        """
+        When null option is enabled and "None" is sent as part of a form to be submitted, it is sent as the
+        string 'null'.  This will check for that condition and gracefully handle the conversion to a NoneType.
+        """
+        if self.null_option is not None and value == settings.FILTERS_NULL_CHOICE_VALUE:
+            return None
+        return super().clean(value)
 
 
 class DynamicModelMultipleChoiceField(DynamicModelChoiceMixin, forms.ModelMultipleChoiceField):

netbox/utilities/forms/widgets.py

@@ -114,6 +114,9 @@ class ContentTypeSelect(StaticSelect2):
 class NumericArrayField(SimpleArrayField):
 
     def to_python(self, value):
+        if not value:
+            return []
+        if isinstance(value, str):
             value = ','.join([str(n) for n in parse_numeric_range(value)])
         return super().to_python(value)
 

upgrade.sh

@@ -29,19 +29,25 @@ eval $COMMAND || {
 # Activate the virtual environment
 source "${VIRTUALENV}/bin/activate"
 
+# Upgrade pip
+COMMAND="pip install --upgrade pip"
+echo "Updating pip ($COMMAND)..."
+eval $COMMAND || exit 1
+pip -V
+
 # Install necessary system packages
-COMMAND="pip3 install wheel"
+COMMAND="pip install wheel"
 echo "Installing Python system packages ($COMMAND)..."
 eval $COMMAND || exit 1
 
 # Install required Python packages
-COMMAND="pip3 install -r requirements.txt"
+COMMAND="pip install -r requirements.txt"
 echo "Installing core dependencies ($COMMAND)..."
 eval $COMMAND || exit 1
 
 # Install optional packages (if any)
 if [ -s "local_requirements.txt" ]; then
-  COMMAND="pip3 install -r local_requirements.txt"
+  COMMAND="pip install -r local_requirements.txt"
   echo "Installing local dependencies ($COMMAND)..."
   eval $COMMAND || exit 1
 elif [ -f "local_requirements.txt" ]; then