Files
netbox/netbox/circuits/filtersets.py
Jason Novinger 7eefb07554
Some checks failed
CI / build (20.x, 3.12) (push) Has been cancelled
CI / build (20.x, 3.13) (push) Has been cancelled
CodeQL / Analyze (${{ matrix.language }}) (none, actions) (push) Has been cancelled
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Has been cancelled
Closes #7604: Add filter modifier dropdowns for advanced lookup operators (#20747)
* Fixes #7604: Add filter modifier dropdowns for advanced lookup operators

Implements dynamic filter modifier UI that allows users to select lookup operators
(exact, contains, starts with, regex, negation, empty/not empty) directly in filter
forms without manual URL parameter editing.

Supports filters for all scalar types and strings, as well as some
related object filters. Explicitly does not support filters on fields
that use APIWidget. That has been broken out in to follow up work.

**Backend:**
- FilterModifierWidget: Wraps form widgets with lookup modifier dropdown
- FilterModifierMixin: Auto-enhances filterset fields with appropriate lookups
- Extended lookup support: Adds negation (n), regex, iregex, empty_true/false lookups
- Field-type-aware: CharField gets text lookups, IntegerField gets comparison operators, etc.

**Frontend:**
- TypeScript handler syncs modifier dropdown with URL parameters
- Dynamically updates form field names (serial → serial__ic) on modifier change
- Flexible-width modifier dropdowns with semantic CSS classes

* Remove extraneous TS comments

* Fix import order

* Fix CircuitFilterForm inheritance

* Enable filter form modifiers on DCIM models

* Enable filter form modifiers on Tenancy models

* Enable filter form modifiers on Wireless models

* Enable filter form modifiers on IPAM models

* Enable filter form modifiers on VPN models

* Enable filter form modifiers on Virtualization models

* Enable filter form modifiers on Circuit models

* Enable filter form modifiers on Users models

* Enable filter form modifiers on Core models

* Enable filter form modifiers on Extras models

* Add ChoiceField support to FilterModifierMixin

Enable filter modifiers for single-choice ChoiceFields in addition to the
existing MultipleChoiceField support. ChoiceFields can now display modifier
dropdowns with "Is", "Is Not", "Is Empty", and "Is Not Empty" options when
the corresponding FilterSet defines those lookups.

The mixin correctly verifies lookup availability against the FilterSet, so
modifiers only appear when multiple lookup options are actually supported.
Currently most FilterSets only define 'exact' for single-choice fields, but
this change enables future FilterSet enhancements to expose additional
lookups for ChoiceFields.

* Address PR feedback: Replace global filterset mappings with registry

* Address PR feedback: Move FilterModifierMixin into base filter form classes

Incorporates FilterModifierMixin into NetBoxModelFilterSetForm and FilterForm,
making filter modifiers automatic for all filter forms throughout the application.

* Fix filter modifier form submission bug with 'action' field collision

Forms with a field named "action" (e.g., ObjectChangeFilterForm) were causing
the form.action property to be shadowed by the field element, resulting in
[object HTMLSelectElement] appearing in the URL path.

Use form.getAttribute('action') instead of form.action to reliably retrieve
the form's action URL without collision from form fields.

Fixes form submission on /core/changelog/ and any other forms with an 'action'
field using filter modifiers.

* Address PR feedback: Move FORM_FIELD_LOOKUPS to module-level constant

Extracts the field type to lookup mappings from FilterModifierMixin class
attribute to a module-level constant for better reusability.

* Address PR feedback: Refactor and consolidate field filtering logic

Consolidated field enhancement logic in FilterModifierMixin by:
- Creating QueryField marker type (CharField subclass) for search fields
- Updating FilterForm and NetBoxModelFilterSetForm to use QueryField for 'q'
- Moving all skip logic into _get_lookup_choices() to return empty list for
  fields that shouldn't be enhanced
- Removing separate _should_skip_field() method
- Removing unused field_name parameter from _get_lookup_choices()
- Replacing hardcoded field name check ('q') with type-based detection

* Address PR feedback: Refactor applied_filters to use FORM_FIELD_LOOKUPS

* Address PR feedback: Rename FilterModifierWidget parameter to widget

* Fix registry pattern to use model identifiers as keys

Changed filterset registration to use model identifiers ('{app_label}.{model_name}')
as registry keys instead of form classes, matching NetBox's pattern for search indexes.

* Address PR feedback: refactor brittle test for APISelect useage

Now checks if widget is actually APISelect, rather than trying to infer
from the class name.

* Refactor register_filterset to be more generic and simple

* Remove unneeded imports left from earlier registry work

* Update app registry for new `filtersets` store

* Remove unused star import, leftover from earlier work

* Enables filter modifiers on APISelect based fields

* Support filter modifiers for ChoiceField

* Include MODIFIER_EMPTY_FALSE/_TRUE in __all__

Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>

* Fix filterset registration for doubly-registered models

* Removed explicit checks against QueryField and [Null]BooleanField

I did add them to FORM_FIELD_LOOKUPS, though, to underscore that they
were considered and are intentially empty for future devs.

* Switch to sentence case for filter pill text

* Fix applied_filters template tag to use field-type-specific lookup labelsresolves

E.g. resolves gt="after" for dates vs "greater than" for numbers

* Verifies that filter pills for exact matches (no lookup
Add test for exact lookup filter pill rendering

* Add guard for FilterModifierWidget with no lookups

* Remove comparison symbols from numeric filter labels

* Match complete tags in widget rendering test assertions

* Check all expected lookups in field enhancement tests

* Move register_filterset to netbox.plugins.registration

* Require registered filterset for filter modifier enhancements

Updates FilterModifierMixin to only enhance form fields when the
associated model has a registered filterset. This provides plugin
safety by ensuring unregistered plugin filtersets fall back to
simple filters without lookup modifiers.

Test changes:
- Create TestModel and TestFilterSet using BaseFilterSet for
automatic lookup generation
- Import dcim.filtersets to ensure Device filterset registration
- Adjust tag field expectations to match actual Device filterset
(has exact/n but not empty lookups)

* Attempt to resolve static conflicts

* Move register_filterset() back to utilities.filtersets

* Add register_filterset() to plugins documentation for filtersets

* Reorder import statements

---------

Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>
2025-12-05 15:13:37 -05:00

601 lines
20 KiB
Python

import django_filters
from django.contrib.contenttypes.models import ContentType
from django.db.models import Q
from django.utils.translation import gettext as _
from dcim.filtersets import CabledObjectFilterSet
from dcim.models import Interface, Location, Region, Site, SiteGroup
from ipam.models import ASN
from netbox.filtersets import NetBoxModelFilterSet, OrganizationalModelFilterSet, PrimaryModelFilterSet
from tenancy.filtersets import ContactModelFilterSet, TenancyFilterSet
from utilities.filters import (
ContentTypeFilter, MultiValueCharFilter, MultiValueNumberFilter, TreeNodeMultipleChoiceFilter,
)
from utilities.filtersets import register_filterset
from .choices import *
from .models import *
__all__ = (
'CircuitFilterSet',
'CircuitGroupAssignmentFilterSet',
'CircuitGroupFilterSet',
'CircuitTerminationFilterSet',
'CircuitTypeFilterSet',
'ProviderNetworkFilterSet',
'ProviderAccountFilterSet',
'ProviderFilterSet',
'VirtualCircuitFilterSet',
'VirtualCircuitTerminationFilterSet',
'VirtualCircuitTypeFilterSet',
)
@register_filterset
class ProviderFilterSet(PrimaryModelFilterSet, ContactModelFilterSet):
region_id = TreeNodeMultipleChoiceFilter(
queryset=Region.objects.all(),
field_name='circuits__terminations___region',
lookup_expr='in',
label=_('Region (ID)'),
)
region = TreeNodeMultipleChoiceFilter(
queryset=Region.objects.all(),
field_name='circuits__terminations___region',
lookup_expr='in',
to_field_name='slug',
label=_('Region (slug)'),
)
site_group_id = TreeNodeMultipleChoiceFilter(
queryset=SiteGroup.objects.all(),
field_name='circuits__terminations___site_group',
lookup_expr='in',
label=_('Site group (ID)'),
)
site_group = TreeNodeMultipleChoiceFilter(
queryset=SiteGroup.objects.all(),
field_name='circuits__terminations___site_group',
lookup_expr='in',
to_field_name='slug',
label=_('Site group (slug)'),
)
site_id = django_filters.ModelMultipleChoiceFilter(
field_name='circuits__terminations___site',
queryset=Site.objects.all(),
label=_('Site'),
)
site = django_filters.ModelMultipleChoiceFilter(
field_name='circuits__terminations___site__slug',
queryset=Site.objects.all(),
to_field_name='slug',
label=_('Site (slug)'),
)
asn_id = django_filters.ModelMultipleChoiceFilter(
field_name='asns',
queryset=ASN.objects.all(),
label=_('ASN (ID)'),
)
asn = django_filters.ModelMultipleChoiceFilter(
field_name='asns__asn',
queryset=ASN.objects.all(),
to_field_name='asn',
label=_('ASN'),
)
class Meta:
model = Provider
fields = ('id', 'name', 'slug', 'description')
def search(self, queryset, name, value):
if not value.strip():
return queryset
return queryset.filter(
Q(name__icontains=value) |
Q(description__icontains=value) |
Q(comments__icontains=value)
)
@register_filterset
class ProviderAccountFilterSet(PrimaryModelFilterSet, ContactModelFilterSet):
provider_id = django_filters.ModelMultipleChoiceFilter(
queryset=Provider.objects.all(),
label=_('Provider (ID)'),
)
provider = django_filters.ModelMultipleChoiceFilter(
field_name='provider__slug',
queryset=Provider.objects.all(),
to_field_name='slug',
label=_('Provider (slug)'),
)
class Meta:
model = ProviderAccount
fields = ('id', 'name', 'account', 'description')
def search(self, queryset, name, value):
if not value.strip():
return queryset
return queryset.filter(
Q(name__icontains=value) |
Q(description__icontains=value) |
Q(account__icontains=value) |
Q(comments__icontains=value)
).distinct()
@register_filterset
class ProviderNetworkFilterSet(PrimaryModelFilterSet):
provider_id = django_filters.ModelMultipleChoiceFilter(
queryset=Provider.objects.all(),
label=_('Provider (ID)'),
)
provider = django_filters.ModelMultipleChoiceFilter(
field_name='provider__slug',
queryset=Provider.objects.all(),
to_field_name='slug',
label=_('Provider (slug)'),
)
class Meta:
model = ProviderNetwork
fields = ('id', 'name', 'service_id', 'description')
def search(self, queryset, name, value):
if not value.strip():
return queryset
return queryset.filter(
Q(name__icontains=value) |
Q(service_id__icontains=value) |
Q(description__icontains=value) |
Q(comments__icontains=value)
).distinct()
@register_filterset
class CircuitTypeFilterSet(OrganizationalModelFilterSet):
class Meta:
model = CircuitType
fields = ('id', 'name', 'slug', 'color', 'description')
@register_filterset
class CircuitFilterSet(PrimaryModelFilterSet, TenancyFilterSet, ContactModelFilterSet):
provider_id = django_filters.ModelMultipleChoiceFilter(
queryset=Provider.objects.all(),
label=_('Provider (ID)'),
)
provider = django_filters.ModelMultipleChoiceFilter(
field_name='provider__slug',
queryset=Provider.objects.all(),
to_field_name='slug',
label=_('Provider (slug)'),
)
provider_account_id = django_filters.ModelMultipleChoiceFilter(
field_name='provider_account',
queryset=ProviderAccount.objects.all(),
label=_('Provider account (ID)'),
)
provider_account = django_filters.ModelMultipleChoiceFilter(
field_name='provider_account__account',
queryset=Provider.objects.all(),
to_field_name='account',
label=_('Provider account (account)'),
)
provider_network_id = django_filters.ModelMultipleChoiceFilter(
field_name='terminations___provider_network',
queryset=ProviderNetwork.objects.all(),
label=_('Provider network (ID)'),
)
type_id = django_filters.ModelMultipleChoiceFilter(
queryset=CircuitType.objects.all(),
label=_('Circuit type (ID)'),
)
type = django_filters.ModelMultipleChoiceFilter(
field_name='type__slug',
queryset=CircuitType.objects.all(),
to_field_name='slug',
label=_('Circuit type (slug)'),
)
status = django_filters.MultipleChoiceFilter(
choices=CircuitStatusChoices,
null_value=None
)
region_id = TreeNodeMultipleChoiceFilter(
queryset=Region.objects.all(),
field_name='terminations___region',
lookup_expr='in',
label=_('Region (ID)'),
)
region = TreeNodeMultipleChoiceFilter(
queryset=Region.objects.all(),
field_name='terminations___region',
lookup_expr='in',
to_field_name='slug',
label=_('Region (slug)'),
)
site_group_id = TreeNodeMultipleChoiceFilter(
queryset=SiteGroup.objects.all(),
field_name='terminations___site_group',
lookup_expr='in',
label=_('Site group (ID)'),
)
site_group = TreeNodeMultipleChoiceFilter(
queryset=SiteGroup.objects.all(),
field_name='terminations___site_group',
lookup_expr='in',
to_field_name='slug',
label=_('Site group (slug)'),
)
site_id = django_filters.ModelMultipleChoiceFilter(
field_name='terminations___site',
queryset=Site.objects.all(),
label=_('Site (ID)'),
)
site = django_filters.ModelMultipleChoiceFilter(
field_name='terminations___site__slug',
queryset=Site.objects.all(),
to_field_name='slug',
label=_('Site (slug)'),
)
location_id = django_filters.ModelMultipleChoiceFilter(
field_name='terminations___location',
label=_('Location (ID)'),
queryset=Location.objects.all(),
)
termination_a_id = django_filters.ModelMultipleChoiceFilter(
queryset=CircuitTermination.objects.all(),
label=_('Termination A (ID)'),
)
termination_z_id = django_filters.ModelMultipleChoiceFilter(
queryset=CircuitTermination.objects.all(),
label=_('Termination A (ID)'),
)
class Meta:
model = Circuit
fields = (
'id', 'cid', 'description', 'install_date', 'termination_date', 'commit_rate', 'distance', 'distance_unit',
)
def search(self, queryset, name, value):
if not value.strip():
return queryset
return queryset.filter(
Q(cid__icontains=value) |
Q(terminations__xconnect_id__icontains=value) |
Q(terminations__pp_info__icontains=value) |
Q(terminations__description__icontains=value) |
Q(description__icontains=value) |
Q(comments__icontains=value)
).distinct()
@register_filterset
class CircuitTerminationFilterSet(NetBoxModelFilterSet, CabledObjectFilterSet):
q = django_filters.CharFilter(
method='search',
label=_('Search'),
)
circuit_id = django_filters.ModelMultipleChoiceFilter(
queryset=Circuit.objects.all(),
label=_('Circuit'),
)
termination_type = ContentTypeFilter()
region_id = TreeNodeMultipleChoiceFilter(
queryset=Region.objects.all(),
field_name='_region',
lookup_expr='in',
label=_('Region (ID)'),
)
region = TreeNodeMultipleChoiceFilter(
queryset=Region.objects.all(),
field_name='_region',
lookup_expr='in',
to_field_name='slug',
label=_('Region (slug)'),
)
site_group_id = TreeNodeMultipleChoiceFilter(
queryset=SiteGroup.objects.all(),
field_name='_site_group',
lookup_expr='in',
label=_('Site group (ID)'),
)
site_group = TreeNodeMultipleChoiceFilter(
queryset=SiteGroup.objects.all(),
field_name='_site_group',
lookup_expr='in',
to_field_name='slug',
label=_('Site group (slug)'),
)
site_id = django_filters.ModelMultipleChoiceFilter(
queryset=Site.objects.all(),
field_name='_site',
label=_('Site (ID)'),
)
site = django_filters.ModelMultipleChoiceFilter(
field_name='_site__slug',
queryset=Site.objects.all(),
to_field_name='slug',
label=_('Site (slug)'),
)
location_id = TreeNodeMultipleChoiceFilter(
queryset=Location.objects.all(),
field_name='_location',
lookup_expr='in',
label=_('Location (ID)'),
)
location = TreeNodeMultipleChoiceFilter(
queryset=Location.objects.all(),
field_name='_location',
lookup_expr='in',
to_field_name='slug',
label=_('Location (slug)'),
)
provider_network_id = django_filters.ModelMultipleChoiceFilter(
queryset=ProviderNetwork.objects.all(),
field_name='_provider_network',
label=_('ProviderNetwork (ID)'),
)
provider_id = django_filters.ModelMultipleChoiceFilter(
field_name='circuit__provider_id',
queryset=Provider.objects.all(),
label=_('Provider (ID)'),
)
provider = django_filters.ModelMultipleChoiceFilter(
field_name='circuit__provider__slug',
queryset=Provider.objects.all(),
to_field_name='slug',
label=_('Provider (slug)'),
)
class Meta:
model = CircuitTermination
fields = (
'id', 'termination_id', 'term_side', 'port_speed', 'upstream_speed', 'xconnect_id', 'description',
'mark_connected', 'pp_info', 'cable_end', 'cable_position',
)
def search(self, queryset, name, value):
if not value.strip():
return queryset
return queryset.filter(
Q(circuit__cid__icontains=value) |
Q(xconnect_id__icontains=value) |
Q(pp_info__icontains=value) |
Q(description__icontains=value)
).distinct()
@register_filterset
class CircuitGroupFilterSet(OrganizationalModelFilterSet, TenancyFilterSet):
class Meta:
model = CircuitGroup
fields = ('id', 'name', 'slug', 'description')
@register_filterset
class CircuitGroupAssignmentFilterSet(NetBoxModelFilterSet):
q = django_filters.CharFilter(
method='search',
label=_('Search'),
)
member_type = ContentTypeFilter()
circuit = MultiValueCharFilter(
method='filter_circuit',
field_name='cid',
label=_('Circuit (CID)'),
)
circuit_id = MultiValueNumberFilter(
method='filter_circuit',
field_name='pk',
label=_('Circuit (ID)'),
)
virtual_circuit = MultiValueCharFilter(
method='filter_virtual_circuit',
field_name='cid',
label=_('Virtual circuit (CID)'),
)
virtual_circuit_id = MultiValueNumberFilter(
method='filter_virtual_circuit',
field_name='pk',
label=_('Virtual circuit (ID)'),
)
provider = MultiValueCharFilter(
method='filter_provider',
field_name='slug',
label=_('Provider (name)'),
)
provider_id = MultiValueNumberFilter(
method='filter_provider',
field_name='pk',
label=_('Provider (ID)'),
)
group_id = django_filters.ModelMultipleChoiceFilter(
queryset=CircuitGroup.objects.all(),
label=_('Circuit group (ID)'),
)
group = django_filters.ModelMultipleChoiceFilter(
field_name='group__slug',
queryset=CircuitGroup.objects.all(),
to_field_name='slug',
label=_('Circuit group (slug)'),
)
class Meta:
model = CircuitGroupAssignment
fields = ('id', 'member_id', 'priority')
def search(self, queryset, name, value):
if not value.strip():
return queryset
return queryset.filter(
Q(member__cid__icontains=value) |
Q(group__name__icontains=value)
)
def filter_circuit(self, queryset, name, value):
circuits = Circuit.objects.filter(**{f'{name}__in': value})
if not circuits.exists():
return queryset.none()
return queryset.filter(
Q(
member_type=ContentType.objects.get_for_model(Circuit),
member_id__in=circuits
)
)
def filter_virtual_circuit(self, queryset, name, value):
virtual_circuits = VirtualCircuit.objects.filter(**{f'{name}__in': value})
if not virtual_circuits.exists():
return queryset.none()
return queryset.filter(
Q(
member_type=ContentType.objects.get_for_model(VirtualCircuit),
member_id__in=virtual_circuits
)
)
def filter_provider(self, queryset, name, value):
providers = Provider.objects.filter(**{f'{name}__in': value})
if not providers.exists():
return queryset.none()
circuits = Circuit.objects.filter(provider__in=providers)
virtual_circuits = VirtualCircuit.objects.filter(provider_network__provider__in=providers)
return queryset.filter(
Q(
member_type=ContentType.objects.get_for_model(Circuit),
member_id__in=circuits
) |
Q(
member_type=ContentType.objects.get_for_model(VirtualCircuit),
member_id__in=virtual_circuits
)
)
@register_filterset
class VirtualCircuitTypeFilterSet(OrganizationalModelFilterSet):
class Meta:
model = VirtualCircuitType
fields = ('id', 'name', 'slug', 'color', 'description')
@register_filterset
class VirtualCircuitFilterSet(PrimaryModelFilterSet, TenancyFilterSet):
provider_id = django_filters.ModelMultipleChoiceFilter(
field_name='provider_network__provider',
queryset=Provider.objects.all(),
label=_('Provider (ID)'),
)
provider = django_filters.ModelMultipleChoiceFilter(
field_name='provider_network__provider__slug',
queryset=Provider.objects.all(),
to_field_name='slug',
label=_('Provider (slug)'),
)
provider_account_id = django_filters.ModelMultipleChoiceFilter(
field_name='provider_account',
queryset=ProviderAccount.objects.all(),
label=_('Provider account (ID)'),
)
provider_account = django_filters.ModelMultipleChoiceFilter(
field_name='provider_account__account',
queryset=Provider.objects.all(),
to_field_name='account',
label=_('Provider account (account)'),
)
provider_network_id = django_filters.ModelMultipleChoiceFilter(
queryset=ProviderNetwork.objects.all(),
label=_('Provider network (ID)'),
)
type_id = django_filters.ModelMultipleChoiceFilter(
queryset=VirtualCircuitType.objects.all(),
label=_('Virtual circuit type (ID)'),
)
type = django_filters.ModelMultipleChoiceFilter(
field_name='type__slug',
queryset=VirtualCircuitType.objects.all(),
to_field_name='slug',
label=_('Virtual circuit type (slug)'),
)
status = django_filters.MultipleChoiceFilter(
choices=CircuitStatusChoices,
null_value=None
)
class Meta:
model = VirtualCircuit
fields = ('id', 'cid', 'description')
def search(self, queryset, name, value):
if not value.strip():
return queryset
return queryset.filter(
Q(cid__icontains=value) |
Q(description__icontains=value) |
Q(comments__icontains=value)
).distinct()
@register_filterset
class VirtualCircuitTerminationFilterSet(NetBoxModelFilterSet):
q = django_filters.CharFilter(
method='search',
label=_('Search'),
)
virtual_circuit_id = django_filters.ModelMultipleChoiceFilter(
queryset=VirtualCircuit.objects.all(),
label=_('Virtual circuit'),
)
role = django_filters.MultipleChoiceFilter(
choices=VirtualCircuitTerminationRoleChoices,
null_value=None
)
provider_id = django_filters.ModelMultipleChoiceFilter(
field_name='virtual_circuit__provider_network__provider',
queryset=Provider.objects.all(),
label=_('Provider (ID)'),
)
provider = django_filters.ModelMultipleChoiceFilter(
field_name='virtual_circuit__provider_network__provider__slug',
queryset=Provider.objects.all(),
to_field_name='slug',
label=_('Provider (slug)'),
)
provider_account_id = django_filters.ModelMultipleChoiceFilter(
field_name='virtual_circuit__provider_account',
queryset=ProviderAccount.objects.all(),
label=_('Provider account (ID)'),
)
provider_account = django_filters.ModelMultipleChoiceFilter(
field_name='virtual_circuit__provider_account__account',
queryset=ProviderAccount.objects.all(),
to_field_name='account',
label=_('Provider account (account)'),
)
provider_network_id = django_filters.ModelMultipleChoiceFilter(
queryset=ProviderNetwork.objects.all(),
field_name='virtual_circuit__provider_network',
label=_('Provider network (ID)'),
)
interface_id = django_filters.ModelMultipleChoiceFilter(
queryset=Interface.objects.all(),
field_name='interface',
label=_('Interface (ID)'),
)
class Meta:
model = VirtualCircuitTermination
fields = ('id', 'interface_id', 'description')
def search(self, queryset, name, value):
if not value.strip():
return queryset
return queryset.filter(
Q(virtual_circuit__cid__icontains=value) |
Q(description__icontains=value)
).distinct()