diff --git a/.github/ISSUE_TEMPLATE/bug_report.yaml b/.github/ISSUE_TEMPLATE/bug_report.yaml index 61d93f286..e04b0e602 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.yaml +++ b/.github/ISSUE_TEMPLATE/bug_report.yaml @@ -17,7 +17,7 @@ body: What version of NetBox are you currently running? (If you don't have access to the most recent NetBox release, consider testing on our [demo instance](https://demo.netbox.dev/) before opening a bug report to see if your issue has already been addressed.) - placeholder: v3.0.1 + placeholder: v3.0.2 validations: required: true - type: dropdown diff --git a/.github/ISSUE_TEMPLATE/feature_request.yaml b/.github/ISSUE_TEMPLATE/feature_request.yaml index 65f452f0b..61a36fb0c 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.yaml +++ b/.github/ISSUE_TEMPLATE/feature_request.yaml @@ -14,7 +14,7 @@ body: attributes: label: NetBox version description: What version of NetBox are you currently running? - placeholder: v3.0.1 + placeholder: v3.0.2 validations: required: true - type: dropdown @@ -30,8 +30,10 @@ body: attributes: label: Proposed functionality description: > - Describe in detail the new feature or behavior you'd like to propose. Include any specific - changes to work flows, data models, or the user interface. + Describe in detail the new feature or behavior you are proposing. Include any specific changes + to work flows, data models, and/or the user interface. The more detail you provide here, the + greater chance your proposal has of being discussed. Feature requests which don't include an + actionable implementation plan will be rejected. validations: required: true - type: textarea diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d734ad2f0..c8e3f47ab 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -58,6 +58,9 @@ jobs: - name: Check UI ESLint, TypeScript, and Prettier Compliance run: yarn --cwd netbox/project-static validate + + - name: Validate Static Asset Integrity + run: scripts/verify-bundles.sh - name: Run tests run: coverage run --source="netbox/" netbox/manage.py test netbox/ diff --git a/.gitignore b/.gitignore index b594efe4b..0ce9a20a8 100644 --- a/.gitignore +++ b/.gitignore @@ -1,16 +1,13 @@ *.pyc *.swp -node_modules npm-debug.log* yarn-debug.log* yarn-error.log* -/netbox/project-static/.cache +/netbox/project-static/node_modules /netbox/project-static/docs/* !/netbox/project-static/docs/.info /netbox/netbox/configuration.py /netbox/netbox/ldap_config.py -/netbox/project-static/.cache -/netbox/project-static/node_modules /netbox/reports/* !/netbox/reports/__init__.py /netbox/scripts/* diff --git a/docs/additional-features/prometheus-metrics.md b/docs/additional-features/prometheus-metrics.md index 56365e336..006ff16a4 100644 --- a/docs/additional-features/prometheus-metrics.md +++ b/docs/additional-features/prometheus-metrics.md @@ -26,4 +26,4 @@ For the exhaustive list of exposed metrics, visit the `/metrics` endpoint on you When deploying NetBox in a multiprocess manner (e.g. running multiple Gunicorn workers) the Prometheus client library requires the use of a shared directory to collect metrics from all worker processes. To configure this, first create or designate a local directory to which the worker processes have read and write access, and then configure your WSGI service (e.g. Gunicorn) to define this path as the `prometheus_multiproc_dir` environment variable. !!! warning - If having accurate long-term metrics in a multiprocess environment is crucial to your deployment, it's recommended you use the `uwsgi` library instead of `gunicorn`. The issue lies in the way `gunicorn` tracks worker processes (vs `uwsgi`) which helps manage the metrics files created by the above configurations. If you're using NetBox with gunicorn in a containerized enviroment following the one-process-per-container methodology, then you will likely not need to change to `uwsgi`. More details can be found in [issue #3779](https://github.com/netbox-community/netbox/issues/3779#issuecomment-590547562). + If having accurate long-term metrics in a multiprocess environment is crucial to your deployment, it's recommended you use the `uwsgi` library instead of `gunicorn`. The issue lies in the way `gunicorn` tracks worker processes (vs `uwsgi`) which helps manage the metrics files created by the above configurations. If you're using NetBox with gunicorn in a containerized environment following the one-process-per-container methodology, then you will likely not need to change to `uwsgi`. More details can be found in [issue #3779](https://github.com/netbox-community/netbox/issues/3779#issuecomment-590547562). diff --git a/docs/development/extending-models.md b/docs/development/extending-models.md index 994d2a040..99c448c06 100644 --- a/docs/development/extending-models.md +++ b/docs/development/extending-models.md @@ -34,11 +34,11 @@ class Foo(models.Model): ## 3. Update relevant querysets -If you're adding a relational field (e.g. `ForeignKey`) and intend to include the data when retreiving a list of objects, be sure to include the field using `prefetch_related()` as appropriate. This will optimize the view and avoid extraneous database queries. +If you're adding a relational field (e.g. `ForeignKey`) and intend to include the data when retrieving a list of objects, be sure to include the field using `prefetch_related()` as appropriate. This will optimize the view and avoid extraneous database queries. ## 4. Update API serializer -Extend the model's API serializer in `.api.serializers` to include the new field. In most cases, it will not be necessary to also extend the nested serializer, which produces a minimal represenation of the model. +Extend the model's API serializer in `.api.serializers` to include the new field. In most cases, it will not be necessary to also extend the nested serializer, which produces a minimal representation of the model. ## 5. Add field to forms diff --git a/docs/graphql-api/overview.md b/docs/graphql-api/overview.md index f1ce4f455..f024306b0 100644 --- a/docs/graphql-api/overview.md +++ b/docs/graphql-api/overview.md @@ -45,7 +45,7 @@ NetBox provides both a singular and plural query field for each object type: * `$OBJECT`: Returns a single object. Must specify the object's unique ID as `(id: 123)`. * `$OBJECT_list`: Returns a list of objects, optionally filtered by given parameters. -For example, query `device(id:123)` to fetch a specific device (identified by its unique ID), and query `device_list` (with an optional set of fitlers) to fetch all devices. +For example, query `device(id:123)` to fetch a specific device (identified by its unique ID), and query `device_list` (with an optional set of filters) to fetch all devices. For more detail on constructing GraphQL queries, see the [Graphene documentation](https://docs.graphene-python.org/en/latest/). diff --git a/docs/installation/3-netbox.md b/docs/installation/3-netbox.md index 43b23a649..d20bfaf6b 100644 --- a/docs/installation/3-netbox.md +++ b/docs/installation/3-netbox.md @@ -70,19 +70,22 @@ If `git` is not already installed, install it: Next, clone the **master** branch of the NetBox GitHub repository into the current directory. (This branch always holds the current stable release.) ```no-highlight -sudo git clone -b master https://github.com/netbox-community/netbox.git . +sudo git clone -b master --depth 1 https://github.com/netbox-community/netbox.git . ``` +!!! note + The `git clone` command above utilizes a "shallow clone" to retrieve only the most recent commit. If you need to download the entire history, omit the `--depth 1` argument. + The `git clone` command should generate output similar to the following: ``` Cloning into '.'... -remote: Counting objects: 1994, done. -remote: Compressing objects: 100% (150/150), done. -remote: Total 1994 (delta 80), reused 0 (delta 0), pack-reused 1842 -Receiving objects: 100% (1994/1994), 472.36 KiB | 0 bytes/s, done. -Resolving deltas: 100% (1495/1495), done. -Checking connectivity... done. +remote: Enumerating objects: 996, done. +remote: Counting objects: 100% (996/996), done. +remote: Compressing objects: 100% (935/935), done. +remote: Total 996 (delta 148), reused 386 (delta 34), pack-reused 0 +Receiving objects: 100% (996/996), 4.26 MiB | 9.81 MiB/s, done. +Resolving deltas: 100% (148/148), done. ``` !!! note diff --git a/docs/installation/4-gunicorn.md b/docs/installation/4-gunicorn.md index 7b56754fe..4fc73a58b 100644 --- a/docs/installation/4-gunicorn.md +++ b/docs/installation/4-gunicorn.md @@ -14,7 +14,7 @@ While the provided configuration should suffice for most initial installations, ## systemd Setup -We'll use systemd to control both gunicorn and NetBox's background worker process. First, copy `contrib/netbox.service` and `contrib/netbox-rq.service` to the `/etc/systemd/system/` directory and reload the systemd dameon: +We'll use systemd to control both gunicorn and NetBox's background worker process. First, copy `contrib/netbox.service` and `contrib/netbox-rq.service` to the `/etc/systemd/system/` directory and reload the systemd daemon: ```no-highlight sudo cp -v /opt/netbox/contrib/*.service /etc/systemd/system/ diff --git a/docs/models/dcim/platform.md b/docs/models/dcim/platform.md index 65188fa6e..347abc5b8 100644 --- a/docs/models/dcim/platform.md +++ b/docs/models/dcim/platform.md @@ -4,6 +4,6 @@ A platform defines the type of software running on a device or virtual machine. Platforms may optionally be limited by manufacturer: If a platform is assigned to a particular manufacturer, it can only be assigned to devices with a type belonging to that manufacturer. -The platform model is also used to indicate which [NAPALM](../../additional-features/napalm.md) driver and any associated arguments NetBox should use when connecting to a remote device. The name of the driver along with optional parameters are stored with the platform. +The platform model is also used to indicate which NAPALM driver (if any) and any associated arguments NetBox should use when connecting to a remote device. The name of the driver along with optional parameters are stored with the platform. The assignment of platforms to devices is an optional feature, and may be disregarded if not desired. diff --git a/docs/release-notes/version-3.0.md b/docs/release-notes/version-3.0.md index 1587d4b43..379a6877e 100644 --- a/docs/release-notes/version-3.0.md +++ b/docs/release-notes/version-3.0.md @@ -1,5 +1,26 @@ # NetBox v3.0 +## v3.0.2 (2021-09-08) + +### Bug Fixes + +* [#7131](https://github.com/netbox-community/netbox/issues/7131) - Fix issue where Site fields were hidden when editing a VLAN group +* [#7148](https://github.com/netbox-community/netbox/issues/7148) - Fix issue where static query parameters with multiple values were not queried properly +* [#7153](https://github.com/netbox-community/netbox/issues/7153) - Allow clearing of assigned device type images +* [#7162](https://github.com/netbox-community/netbox/issues/7162) - Ensure consistent treatment of `BASE_PATH` for UI-driven API requests +* [#7164](https://github.com/netbox-community/netbox/issues/7164) - Fix styling of "decommissioned" label for circuits +* [#7169](https://github.com/netbox-community/netbox/issues/7169) - Fix CSV import file upload +* [#7176](https://github.com/netbox-community/netbox/issues/7176) - Fix issue where query parameters were duplicated across different forms of the same type +* [#7179](https://github.com/netbox-community/netbox/issues/7179) - Prevent obscuring "connect" pop-up for interfaces under device view +* [#7188](https://github.com/netbox-community/netbox/issues/7188) - Fix issue where select fields with `null_option` did not render or send the null option +* [#7189](https://github.com/netbox-community/netbox/issues/7189) - Set connection factory for django-redis when Sentinel is in use +* [#7191](https://github.com/netbox-community/netbox/issues/7191) - Fix issue where API-backed multi-select elements cleared selected options when adding new options +* [#7193](https://github.com/netbox-community/netbox/issues/7193) - Fix prefix (flat) template issue when viewing child prefixes with prefixes available +* [#7205](https://github.com/netbox-community/netbox/issues/7205) - Fix issue where selected fields with `null_option` set were not added to applied filters +* [#7209](https://github.com/netbox-community/netbox/issues/7209) - Allow unlimited API results when `MAX_PAGE_SIZE` is disabled + +--- + ## v3.0.1 (2021-09-01) ### Bug Fixes diff --git a/netbox/circuits/choices.py b/netbox/circuits/choices.py index bbf536800..0efa431fa 100644 --- a/netbox/circuits/choices.py +++ b/netbox/circuits/choices.py @@ -29,7 +29,7 @@ class CircuitStatusChoices(ChoiceSet): STATUS_PLANNED: 'info', STATUS_PROVISIONING: 'primary', STATUS_OFFLINE: 'danger', - STATUS_DECOMMISSIONED: 'default', + STATUS_DECOMMISSIONED: 'secondary', } diff --git a/netbox/dcim/forms.py b/netbox/dcim/forms.py index 367980ac4..c1f8eccf8 100644 --- a/netbox/dcim/forms.py +++ b/netbox/dcim/forms.py @@ -23,10 +23,10 @@ from tenancy.forms import TenancyFilterForm, TenancyForm from tenancy.models import Tenant from utilities.forms import ( APISelect, APISelectMultiple, add_blank_choice, BootstrapMixin, BulkEditForm, BulkEditNullBooleanSelect, - ColorField, CommentField, CSVChoiceField, CSVContentTypeField, CSVModelChoiceField, CSVTypedChoiceField, - DynamicModelChoiceField, DynamicModelMultipleChoiceField, ExpandableNameField, form_from_model, JSONField, - NumericArrayField, SelectWithPK, SmallTextarea, SlugField, StaticSelect, StaticSelectMultiple, TagFilterField, - BOOLEAN_WITH_BLANK_CHOICES, + ClearableFileInput, ColorField, CommentField, CSVChoiceField, CSVContentTypeField, CSVModelChoiceField, + CSVTypedChoiceField, DynamicModelChoiceField, DynamicModelMultipleChoiceField, ExpandableNameField, form_from_model, + JSONField, NumericArrayField, SelectWithPK, SmallTextarea, SlugField, StaticSelect, StaticSelectMultiple, + TagFilterField, BOOLEAN_WITH_BLANK_CHOICES, ) from virtualization.models import Cluster, ClusterGroup from .choices import * @@ -1271,10 +1271,10 @@ class DeviceTypeForm(BootstrapMixin, CustomFieldModelForm): ) widgets = { 'subdevice_role': StaticSelect(), - 'front_image': forms.ClearableFileInput(attrs={ + 'front_image': ClearableFileInput(attrs={ 'accept': DEVICETYPE_IMAGE_FORMATS }), - 'rear_image': forms.ClearableFileInput(attrs={ + 'rear_image': ClearableFileInput(attrs={ 'accept': DEVICETYPE_IMAGE_FORMATS }) } diff --git a/netbox/extras/migrations/0062_clear_secrets_changelog.py b/netbox/extras/migrations/0062_clear_secrets_changelog.py new file mode 100644 index 000000000..e76fc8d34 --- /dev/null +++ b/netbox/extras/migrations/0062_clear_secrets_changelog.py @@ -0,0 +1,26 @@ +from django.db import migrations + + +def clear_secrets_changelog(apps, schema_editor): + """ + Delete all ObjectChange records referencing a model within the old secrets app (pre-v3.0). + """ + ContentType = apps.get_model('contenttypes', 'ContentType') + ObjectChange = apps.get_model('extras', 'ObjectChange') + + content_type_ids = ContentType.objects.filter(app_label='secrets').values_list('id', flat=True) + ObjectChange.objects.filter(changed_object_type__in=content_type_ids).delete() + + +class Migration(migrations.Migration): + + dependencies = [ + ('extras', '0061_extras_change_logging'), + ] + + operations = [ + migrations.RunPython( + code=clear_secrets_changelog, + reverse_code=migrations.RunPython.noop + ), + ] diff --git a/netbox/ipam/tables.py b/netbox/ipam/tables.py index e4bb7d693..5afb6a9c9 100644 --- a/netbox/ipam/tables.py +++ b/netbox/ipam/tables.py @@ -25,6 +25,15 @@ PREFIX_LINK = """ {{ record.prefix }} """ +PREFIXFLAT_LINK = """ +{% load helpers %} +{% if record.pk %} + {{ record.prefix }} +{% else %} + — +{% endif %} +""" + PREFIX_ROLE_LINK = """ {% if record.role %} {{ record.role }} @@ -281,10 +290,10 @@ class PrefixTable(BaseTable): template_code=PREFIX_LINK, attrs={'td': {'class': 'text-nowrap'}} ) - prefix_flat = tables.Column( - accessor=Accessor('prefix'), - linkify=True, - verbose_name='Prefix (Flat)' + prefix_flat = tables.TemplateColumn( + template_code=PREFIXFLAT_LINK, + attrs={'td': {'class': 'text-nowrap'}}, + verbose_name='Prefix (Flat)', ) depth = tables.Column( accessor=Accessor('_depth'), diff --git a/netbox/netbox/api/pagination.py b/netbox/netbox/api/pagination.py index 77af755ce..e34cb27d0 100644 --- a/netbox/netbox/api/pagination.py +++ b/netbox/netbox/api/pagination.py @@ -34,13 +34,22 @@ class OptionalLimitOffsetPagination(LimitOffsetPagination): return list(queryset[self.offset:]) def get_limit(self, request): - limit = super().get_limit(request) + if self.limit_query_param: + try: + limit = int(request.query_params[self.limit_query_param]) + if limit < 0: + raise ValueError() + # Enforce maximum page size, if defined + if settings.MAX_PAGE_SIZE: + if limit == 0: + return settings.MAX_PAGE_SIZE + else: + return min(limit, settings.MAX_PAGE_SIZE) + return limit + except (KeyError, ValueError): + pass - # Enforce maximum page size - if settings.MAX_PAGE_SIZE: - limit = min(limit, settings.MAX_PAGE_SIZE) - - return limit + return self.default_limit def get_next_link(self): diff --git a/netbox/netbox/settings.py b/netbox/netbox/settings.py index f70be12a0..416536654 100644 --- a/netbox/netbox/settings.py +++ b/netbox/netbox/settings.py @@ -16,7 +16,7 @@ from django.core.validators import URLValidator # Environment setup # -VERSION = '3.0.1' +VERSION = '3.0.2' # Hostname HOSTNAME = platform.node() @@ -250,6 +250,7 @@ CACHES = { } } if CACHING_REDIS_SENTINELS: + DJANGO_REDIS_CONNECTION_FACTORY = 'django_redis.pool.SentinelConnectionFactory' CACHES['default']['LOCATION'] = f'{CACHING_REDIS_PROTO}://{CACHING_REDIS_SENTINEL_SERVICE}/{CACHING_REDIS_DATABASE}' CACHES['default']['OPTIONS']['CLIENT_CLASS'] = 'django_redis.client.SentinelClient' CACHES['default']['OPTIONS']['SENTINELS'] = CACHING_REDIS_SENTINELS diff --git a/netbox/netbox/views/generic.py b/netbox/netbox/views/generic.py index aafb2f3d8..0d4f89d0f 100644 --- a/netbox/netbox/views/generic.py +++ b/netbox/netbox/views/generic.py @@ -21,8 +21,7 @@ from extras.signals import clear_webhooks from utilities.error_handlers import handle_protectederror from utilities.exceptions import AbortTransaction, PermissionsViolation from utilities.forms import ( - BootstrapMixin, BulkRenameForm, ConfirmationForm, CSVDataField, CSVFileField, ImportForm, TableConfigForm, - restrict_form_fields, + BootstrapMixin, BulkRenameForm, ConfirmationForm, CSVDataField, CSVFileField, ImportForm, restrict_form_fields, ) from utilities.permissions import get_permission_for_model from utilities.tables import paginate_table diff --git a/netbox/project-static/dist/config.js b/netbox/project-static/dist/config.js index cf1022589..7cf3ccb30 100644 Binary files a/netbox/project-static/dist/config.js and b/netbox/project-static/dist/config.js differ diff --git a/netbox/project-static/dist/config.js.map b/netbox/project-static/dist/config.js.map index 5f27e84a6..d070a0451 100644 Binary files a/netbox/project-static/dist/config.js.map and b/netbox/project-static/dist/config.js.map differ diff --git a/netbox/project-static/dist/jobs.js b/netbox/project-static/dist/jobs.js index 3faf4b7ec..2aedf1219 100644 Binary files a/netbox/project-static/dist/jobs.js and b/netbox/project-static/dist/jobs.js differ diff --git a/netbox/project-static/dist/jobs.js.map b/netbox/project-static/dist/jobs.js.map index e07a4157d..d7c1dbcbf 100644 Binary files a/netbox/project-static/dist/jobs.js.map and b/netbox/project-static/dist/jobs.js.map differ diff --git a/netbox/project-static/dist/lldp.js b/netbox/project-static/dist/lldp.js index 86adb3db6..7fac1012a 100644 Binary files a/netbox/project-static/dist/lldp.js and b/netbox/project-static/dist/lldp.js differ diff --git a/netbox/project-static/dist/lldp.js.map b/netbox/project-static/dist/lldp.js.map index 028c35995..911cd77c3 100644 Binary files a/netbox/project-static/dist/lldp.js.map and b/netbox/project-static/dist/lldp.js.map differ diff --git a/netbox/project-static/dist/netbox-dark.css b/netbox/project-static/dist/netbox-dark.css index 4caf6a4e6..196966fea 100644 Binary files a/netbox/project-static/dist/netbox-dark.css and b/netbox/project-static/dist/netbox-dark.css differ diff --git a/netbox/project-static/dist/netbox-light.css b/netbox/project-static/dist/netbox-light.css index f5db11538..b915a414f 100644 Binary files a/netbox/project-static/dist/netbox-light.css and b/netbox/project-static/dist/netbox-light.css differ diff --git a/netbox/project-static/dist/netbox-print.css b/netbox/project-static/dist/netbox-print.css index 170e92bc4..413c4d0ad 100644 Binary files a/netbox/project-static/dist/netbox-print.css and b/netbox/project-static/dist/netbox-print.css differ diff --git a/netbox/project-static/dist/netbox.js b/netbox/project-static/dist/netbox.js index c64df2e26..33d004108 100644 Binary files a/netbox/project-static/dist/netbox.js and b/netbox/project-static/dist/netbox.js differ diff --git a/netbox/project-static/dist/netbox.js.map b/netbox/project-static/dist/netbox.js.map index c9deea0b1..8bee87505 100644 Binary files a/netbox/project-static/dist/netbox.js.map and b/netbox/project-static/dist/netbox.js.map differ diff --git a/netbox/project-static/dist/status.js b/netbox/project-static/dist/status.js index 2f3c2f762..c817efb4c 100644 Binary files a/netbox/project-static/dist/status.js and b/netbox/project-static/dist/status.js differ diff --git a/netbox/project-static/dist/status.js.map b/netbox/project-static/dist/status.js.map index fc612cec8..53b01423e 100644 Binary files a/netbox/project-static/dist/status.js.map and b/netbox/project-static/dist/status.js.map differ diff --git a/netbox/project-static/src/buttons/connectionToggle.ts b/netbox/project-static/src/buttons/connectionToggle.ts index 6485bbb50..74b32dc3a 100644 --- a/netbox/project-static/src/buttons/connectionToggle.ts +++ b/netbox/project-static/src/buttons/connectionToggle.ts @@ -8,12 +8,12 @@ import { isTruthy, apiPatch, hasError, getElements } from '../util'; * @param element Connection Toggle Button Element */ function toggleConnection(element: HTMLButtonElement): void { - const id = element.getAttribute('data'); + const url = element.getAttribute('data-url'); const connected = element.classList.contains('connected'); const status = connected ? 'planned' : 'connected'; - if (isTruthy(id)) { - apiPatch(`/api/dcim/cables/${id}/`, { status }).then(res => { + if (isTruthy(url)) { + apiPatch(url, { status }).then(res => { if (hasError(res)) { // If the API responds with an error, show it to the user. createToast('danger', 'Error', res.error).show(); diff --git a/netbox/project-static/src/forms/scopeSelector.ts b/netbox/project-static/src/forms/scopeSelector.ts index 58163bb19..e1c7014bf 100644 --- a/netbox/project-static/src/forms/scopeSelector.ts +++ b/netbox/project-static/src/forms/scopeSelector.ts @@ -1,8 +1,21 @@ import { getElements, toggleVisibility } from '../util'; type ShowHideMap = { - default: { hide: string[]; show: string[] }; - [k: string]: { hide: string[]; show: string[] }; + /** + * Name of view to which this map should apply. + * + * @example vlangroup_edit + */ + [view: string]: { + /** + * Default layout. + */ + default: { hide: string[]; show: string[] }; + /** + * Field name to layout mapping. + */ + [fieldName: string]: { hide: string[]; show: string[] }; + }; }; /** @@ -14,45 +27,47 @@ type ShowHideMap = { * showHideMap.region.show should be shown. */ const showHideMap: ShowHideMap = { - region: { - hide: ['id_sitegroup', 'id_site', 'id_location', 'id_rack', 'id_clustergroup', 'id_cluster'], - show: ['id_region'], - }, - 'site group': { - hide: ['id_region', 'id_site', 'id_location', 'id_rack', 'id_clustergroup', 'id_cluster'], - show: ['id_sitegroup'], - }, - site: { - hide: ['id_location', 'id_rack', 'id_clustergroup', 'id_cluster'], - show: ['id_region', 'id_sitegroup', 'id_site'], - }, - location: { - hide: ['id_rack', 'id_clustergroup', 'id_cluster'], - show: ['id_region', 'id_sitegroup', 'id_site', 'id_location'], - }, - rack: { - hide: ['id_clustergroup', 'id_cluster'], - show: ['id_region', 'id_sitegroup', 'id_site', 'id_location', 'id_rack'], - }, - 'cluster group': { - hide: ['id_region', 'id_sitegroup', 'id_site', 'id_location', 'id_rack', 'id_cluster'], - show: ['id_clustergroup'], - }, - cluster: { - hide: ['id_region', 'id_sitegroup', 'id_site', 'id_location', 'id_rack'], - show: ['id_clustergroup', 'id_cluster'], - }, - default: { - hide: [ - 'id_region', - 'id_sitegroup', - 'id_site', - 'id_location', - 'id_rack', - 'id_clustergroup', - 'id_cluster', - ], - show: [], + vlangroup_edit: { + region: { + hide: ['id_sitegroup', 'id_site', 'id_location', 'id_rack', 'id_clustergroup', 'id_cluster'], + show: ['id_region'], + }, + 'site group': { + hide: ['id_region', 'id_site', 'id_location', 'id_rack', 'id_clustergroup', 'id_cluster'], + show: ['id_sitegroup'], + }, + site: { + hide: ['id_location', 'id_rack', 'id_clustergroup', 'id_cluster'], + show: ['id_region', 'id_sitegroup', 'id_site'], + }, + location: { + hide: ['id_rack', 'id_clustergroup', 'id_cluster'], + show: ['id_region', 'id_sitegroup', 'id_site', 'id_location'], + }, + rack: { + hide: ['id_clustergroup', 'id_cluster'], + show: ['id_region', 'id_sitegroup', 'id_site', 'id_location', 'id_rack'], + }, + 'cluster group': { + hide: ['id_region', 'id_sitegroup', 'id_site', 'id_location', 'id_rack', 'id_cluster'], + show: ['id_clustergroup'], + }, + cluster: { + hide: ['id_region', 'id_sitegroup', 'id_site', 'id_location', 'id_rack'], + show: ['id_clustergroup', 'id_cluster'], + }, + default: { + hide: [ + 'id_region', + 'id_sitegroup', + 'id_site', + 'id_location', + 'id_rack', + 'id_clustergroup', + 'id_cluster', + ], + show: [], + }, }, }; /** @@ -76,11 +91,11 @@ function toggleParentVisibility(query: string, action: 'show' | 'hide') { /** * Handle changes to the Scope Type field. */ -function handleScopeChange(element: HTMLSelectElement) { +function handleScopeChange

(view: P, element: HTMLSelectElement) { // Scope type's innerText looks something like `DCIM > region`. const scopeType = element.options[element.selectedIndex].innerText.toLowerCase(); - for (const [scope, fields] of Object.entries(showHideMap)) { + for (const [scope, fields] of Object.entries(showHideMap[view])) { // If the scope type ends with the specified scope, toggle its field visibility according to // the show/hide values. if (scopeType.endsWith(scope)) { @@ -94,7 +109,7 @@ function handleScopeChange(element: HTMLSelectElement) { break; } else { // Otherwise, hide all fields. - for (const field of showHideMap.default.hide) { + for (const field of showHideMap[view].default.hide) { toggleParentVisibility(`#${field}`, 'hide'); } } @@ -105,8 +120,12 @@ function handleScopeChange(element: HTMLSelectElement) { * Initialize scope type select event listeners. */ export function initScopeSelector(): void { - for (const element of getElements('#id_scope_type')) { - handleScopeChange(element); - element.addEventListener('change', () => handleScopeChange(element)); + for (const view of Object.keys(showHideMap)) { + for (const element of getElements( + `html[data-netbox-url-name="${view}"] #id_scope_type`, + )) { + handleScopeChange(view, element); + element.addEventListener('change', () => handleScopeChange(view, element)); + } } } diff --git a/netbox/project-static/src/jobs.ts b/netbox/project-static/src/jobs.ts index 8a8a3fd12..dedf0706d 100644 --- a/netbox/project-static/src/jobs.ts +++ b/netbox/project-static/src/jobs.ts @@ -4,7 +4,7 @@ import { apiGetBase, hasError, getNetboxData } from './util'; let timeout: number = 1000; interface JobInfo { - id: Nullable; + url: Nullable; complete: boolean; } @@ -23,15 +23,16 @@ function asyncTimeout(ms: number) { function getJobInfo(): JobInfo { let complete = false; - const id = getNetboxData('data-job-id'); - const jobComplete = getNetboxData('data-job-complete'); + // Determine the API URL for the job status + const url = getNetboxData('data-job-url'); // Determine the job completion status, if present. If the job is not complete, the value will be // "None". Otherwise, it will be a stringified date. + const jobComplete = getNetboxData('data-job-complete'); if (typeof jobComplete === 'string' && jobComplete.toLowerCase() !== 'none') { complete = true; } - return { id, complete }; + return { url, complete }; } /** @@ -59,10 +60,10 @@ function updateLabel(status: JobStatus) { /** * Recursively check the job's status. - * @param id Job ID + * @param url API URL for job result */ -async function checkJobStatus(id: string) { - const res = await apiGetBase(`/api/extras/job-results/${id}/`); +async function checkJobStatus(url: string) { + const res = await apiGetBase(url); if (hasError(res)) { // If the response is an API error, display an error message and stop checking for job status. const toast = createToast('danger', 'Error', res.error); @@ -82,17 +83,17 @@ async function checkJobStatus(id: string) { if (timeout < 10000) { timeout += 1000; } - await Promise.all([checkJobStatus(id), asyncTimeout(timeout)]); + await Promise.all([checkJobStatus(url), asyncTimeout(timeout)]); } } } function initJobs() { - const { id, complete } = getJobInfo(); + const { url, complete } = getJobInfo(); - if (id !== null && !complete) { + if (url !== null && !complete) { // If there is a job ID and it is not completed, check for the job's status. - Promise.resolve(checkJobStatus(id)); + Promise.resolve(checkJobStatus(url)); } } diff --git a/netbox/project-static/src/select/api/apiSelect.ts b/netbox/project-static/src/select/api/apiSelect.ts index fe7c218a3..6d83daaac 100644 --- a/netbox/project-static/src/select/api/apiSelect.ts +++ b/netbox/project-static/src/select/api/apiSelect.ts @@ -5,7 +5,7 @@ import SlimSelect from 'slim-select'; import { createToast } from '../../bs'; import { hasUrl, hasExclusions, isTrigger } from '../util'; import { DynamicParamsMap } from './dynamicParams'; -import { isStaticParams } from './types'; +import { isStaticParams, isOption } from './types'; import { hasMore, isTruthy, @@ -23,7 +23,7 @@ import type { Option } from 'slim-select/dist/data'; import type { Trigger, PathFilter, ApplyMethod, QueryFilter } from './types'; // Empty placeholder option. -const PLACEHOLDER = { +const EMPTY_PLACEHOLDER = { value: '', text: '', placeholder: true, @@ -52,6 +52,18 @@ export class APISelect { */ public readonly placeholder: string; + /** + * Empty/placeholder option. Display text is optionally overridden via the `data-empty-option` + * attribute. + */ + public readonly emptyOption: Option; + + /** + * Null option. When `data-null-option` attribute is a string, the value is used to created an + * option of type `{text: '': 'null'}`. + */ + public readonly nullOption: Nullable

{% if perms.dcim.change_interface %} diff --git a/netbox/templates/dcim/inc/cable_toggle_buttons.html b/netbox/templates/dcim/inc/cable_toggle_buttons.html index 3fe34acd7..77db3093f 100644 --- a/netbox/templates/dcim/inc/cable_toggle_buttons.html +++ b/netbox/templates/dcim/inc/cable_toggle_buttons.html @@ -1,10 +1,10 @@ {% if perms.dcim.change_cable %} {% if cable.status == 'connected' %} - {% else %} - {% endif %} diff --git a/netbox/templates/extras/report_result.html b/netbox/templates/extras/report_result.html index 1f757d4cb..f8da50e8e 100644 --- a/netbox/templates/extras/report_result.html +++ b/netbox/templates/extras/report_result.html @@ -96,6 +96,6 @@ {% endblock %} {% block data %} - + {% endblock %} diff --git a/netbox/templates/extras/script_result.html b/netbox/templates/extras/script_result.html index 59ca8a69d..f463b0f2c 100644 --- a/netbox/templates/extras/script_result.html +++ b/netbox/templates/extras/script_result.html @@ -112,6 +112,6 @@ {% endblock content-wrapper %} {% block data %} - + {% endblock %} diff --git a/netbox/templates/generic/object_bulk_import.html b/netbox/templates/generic/object_bulk_import.html index d1f744795..2d5957268 100644 --- a/netbox/templates/generic/object_bulk_import.html +++ b/netbox/templates/generic/object_bulk_import.html @@ -17,17 +17,32 @@ {% block content %}
-
- {% csrf_token %} - {% render_form form %} -
-
- {% if return_url %} - Cancel - {% endif %} - -
+
    + + +
+ + {% csrf_token %} +
+
+ {% render_field form.csv %}
+
+ {% render_field form.csv_file %} +
+
+
+
+ {% if return_url %} + Cancel + {% endif %} + +
+
{% if fields %}
diff --git a/netbox/templates/utilities/render_field.html b/netbox/templates/utilities/render_field.html index a8ea9c25f..c32ca4e4a 100644 --- a/netbox/templates/utilities/render_field.html +++ b/netbox/templates/utilities/render_field.html @@ -86,18 +86,28 @@
-{% elif field|widget_type == 'fileinput' or field|widget_type == 'clearablefileinput' %} -
- - +{% elif field|widget_type == 'fileinput' %} +
+ + +
+ +{% elif field|widget_type == 'clearablefileinput' %} +
+ +
+ {{ field }} +
{% elif field|widget_type == 'selectmultiple' %} diff --git a/netbox/templates/utilities/templatetags/table_config_form.html b/netbox/templates/utilities/templatetags/table_config_form.html index 5e17497e9..cad3a306a 100644 --- a/netbox/templates/utilities/templatetags/table_config_form.html +++ b/netbox/templates/utilities/templatetags/table_config_form.html @@ -7,7 +7,7 @@
Table Configuration
-
+
{{ form.available_columns.label }} diff --git a/netbox/utilities/forms/fields.py b/netbox/utilities/forms/fields.py index 2d1355250..2561c2e22 100644 --- a/netbox/utilities/forms/fields.py +++ b/netbox/utilities/forms/fields.py @@ -376,7 +376,7 @@ class DynamicModelChoiceMixin: widget = widgets.APISelect def __init__(self, query_params=None, initial_params=None, null_option=None, disabled_indicator=None, fetch_trigger=None, - *args, **kwargs): + empty_label=None, *args, **kwargs): self.query_params = query_params or {} self.initial_params = initial_params or {} self.null_option = null_option @@ -386,11 +386,14 @@ class DynamicModelChoiceMixin: # to_field_name is set by ModelChoiceField.__init__(), but we need to set it early for reference # by widget_attrs() self.to_field_name = kwargs.get('to_field_name') + self.empty_option = empty_label or "" super().__init__(*args, **kwargs) def widget_attrs(self, widget): - attrs = {} + attrs = { + 'data-empty-option': self.empty_option + } # Set value-field attribute if the field specifies to_field_name if self.to_field_name: @@ -474,3 +477,13 @@ class DynamicModelMultipleChoiceField(DynamicModelChoiceMixin, forms.ModelMultip """ filter = django_filters.ModelMultipleChoiceFilter widget = widgets.APISelectMultiple + + def clean(self, value): + """ + When null option is enabled and "None" is sent as part of a form to be submitted, it is sent as the + string 'null'. This will check for that condition and gracefully handle the conversion to a NoneType. + """ + if self.null_option is not None and settings.FILTERS_NULL_CHOICE_VALUE in value: + value = [v for v in value if v != settings.FILTERS_NULL_CHOICE_VALUE] + return [None, *value] + return super().clean(value) diff --git a/netbox/utilities/forms/forms.py b/netbox/utilities/forms/forms.py index 2073d6a94..8bc113218 100644 --- a/netbox/utilities/forms/forms.py +++ b/netbox/utilities/forms/forms.py @@ -4,7 +4,7 @@ import re import yaml from django import forms -from .widgets import APISelect, APISelectMultiple, StaticSelect +from .widgets import APISelect, APISelectMultiple, ClearableFileInput, StaticSelect __all__ = ( @@ -29,12 +29,12 @@ class BootstrapMixin(forms.BaseForm): exempt_widgets = [ forms.CheckboxInput, - forms.ClearableFileInput, forms.FileInput, forms.RadioSelect, forms.Select, APISelect, APISelectMultiple, + ClearableFileInput, StaticSelect, ] diff --git a/netbox/utilities/forms/utils.py b/netbox/utilities/forms/utils.py index 0121b250c..bb1f56c4d 100644 --- a/netbox/utilities/forms/utils.py +++ b/netbox/utilities/forms/utils.py @@ -1,6 +1,7 @@ import re from django import forms +from django.conf import settings from django.forms.models import fields_for_model from utilities.choices import unpack_grouped_choices @@ -120,13 +121,20 @@ def get_selected_values(form, field_name): if not hasattr(form, 'cleaned_data'): form.is_valid() filter_data = form.cleaned_data.get(field_name) - + field = form.fields[field_name] # Selection field - if hasattr(form.fields[field_name], 'choices'): + if hasattr(field, 'choices'): try: - choices = dict(unpack_grouped_choices(form.fields[field_name].choices)) + choices = unpack_grouped_choices(field.choices) + + if hasattr(field, 'null_option'): + # If the field has a `null_option` attribute set and it is selected, + # add it to the field's grouped choices. + if field.null_option is not None and None in filter_data: + choices.append((settings.FILTERS_NULL_CHOICE_VALUE, field.null_option)) + return [ - label for value, label in choices.items() if str(value) in filter_data + label for value, label in choices if str(value) in filter_data or None in filter_data ] except TypeError: # Field uses dynamic choices. Show all that have been populated. diff --git a/netbox/utilities/forms/widgets.py b/netbox/utilities/forms/widgets.py index 1625da623..069cbf8db 100644 --- a/netbox/utilities/forms/widgets.py +++ b/netbox/utilities/forms/widgets.py @@ -12,6 +12,7 @@ __all__ = ( 'APISelect', 'APISelectMultiple', 'BulkEditNullBooleanSelect', + 'ClearableFileInput', 'ColorSelect', 'ContentTypeSelect', 'DatePicker', @@ -135,6 +136,13 @@ class NumericArrayField(SimpleArrayField): return super().to_python(value) +class ClearableFileInput(forms.ClearableFileInput): + """ + Override Django's stock ClearableFileInput with a custom template. + """ + template_name = 'widgets/clearable_file_input.html' + + class APISelect(SelectWithDisabled): """ A select widget populated via an API call @@ -155,6 +163,13 @@ class APISelect(SelectWithDisabled): if api_url: self.attrs['data-url'] = '/{}{}'.format(settings.BASE_PATH, api_url.lstrip('/')) # Inject BASE_PATH + def __deepcopy__(self, memo): + """Reset `static_params` and `dynamic_params` when APISelect is deepcopied.""" + result = super().__deepcopy__(memo) + result.dynamic_params = {} + result.static_params = {} + return result + def _process_query_param(self, key: str, value: JSONPrimitive) -> None: """ Based on query param value's type and value, update instance's dynamic/static params. diff --git a/netbox/utilities/templates/widgets/clearable_file_input.html b/netbox/utilities/templates/widgets/clearable_file_input.html new file mode 100644 index 000000000..9b7f980d4 --- /dev/null +++ b/netbox/utilities/templates/widgets/clearable_file_input.html @@ -0,0 +1,24 @@ +
+
+ {% if widget.is_initial %} + {{ widget.value }} + {% if not widget.required %} +
+ + + {% endif %} + {% else %} + None assigned + {% endif %} +
+
+ +
+
diff --git a/netbox/utilities/testing/api.py b/netbox/utilities/testing/api.py index c67296c47..8498f93ae 100644 --- a/netbox/utilities/testing/api.py +++ b/netbox/utilities/testing/api.py @@ -39,13 +39,13 @@ class APITestCase(ModelTestCase): def setUp(self): """ - Create a superuser and token for API calls. + Create a user and token for API calls. """ # Create the test user and assign permissions self.user = User.objects.create_user(username='testuser') self.add_permissions(*self.user_permissions) self.token = Token.objects.create(user=self.user) - self.header = {'HTTP_AUTHORIZATION': 'Token {}'.format(self.token.key)} + self.header = {'HTTP_AUTHORIZATION': f'Token {self.token.key}'} def _get_view_namespace(self): return f'{self.view_namespace or self.model._meta.app_label}-api' diff --git a/netbox/utilities/tests/test_api.py b/netbox/utilities/tests/test_api.py index 2cc9accaa..5b711056a 100644 --- a/netbox/utilities/tests/test_api.py +++ b/netbox/utilities/tests/test_api.py @@ -1,7 +1,8 @@ import urllib.parse +from django.conf import settings from django.contrib.contenttypes.models import ContentType -from django.test import Client, TestCase +from django.test import Client, TestCase, override_settings from django.urls import reverse from rest_framework import status @@ -122,6 +123,59 @@ class WritableNestedSerializerTest(APITestCase): self.assertEqual(VLAN.objects.count(), 0) +class APIPaginationTestCase(APITestCase): + user_permissions = ('dcim.view_site',) + + @classmethod + def setUpTestData(cls): + cls.url = reverse('dcim-api:site-list') + + # Create a large number of Sites for testing + Site.objects.bulk_create([ + Site(name=f'Site {i}', slug=f'site-{i}') for i in range(1, 101) + ]) + + def test_default_page_size(self): + response = self.client.get(self.url, format='json', **self.header) + page_size = settings.PAGINATE_COUNT + self.assertLess(page_size, 100, "Default page size not sufficient for data set") + + self.assertHttpStatus(response, status.HTTP_200_OK) + self.assertEqual(response.data['count'], 100) + self.assertTrue(response.data['next'].endswith(f'?limit={page_size}&offset={page_size}')) + self.assertIsNone(response.data['previous']) + self.assertEqual(len(response.data['results']), page_size) + + def test_custom_page_size(self): + response = self.client.get(f'{self.url}?limit=10', format='json', **self.header) + + self.assertHttpStatus(response, status.HTTP_200_OK) + self.assertEqual(response.data['count'], 100) + self.assertTrue(response.data['next'].endswith(f'?limit=10&offset=10')) + self.assertIsNone(response.data['previous']) + self.assertEqual(len(response.data['results']), 10) + + @override_settings(MAX_PAGE_SIZE=20) + def test_max_page_size(self): + response = self.client.get(f'{self.url}?limit=0', format='json', **self.header) + + self.assertHttpStatus(response, status.HTTP_200_OK) + self.assertEqual(response.data['count'], 100) + self.assertTrue(response.data['next'].endswith(f'?limit=20&offset=20')) + self.assertIsNone(response.data['previous']) + self.assertEqual(len(response.data['results']), 20) + + @override_settings(MAX_PAGE_SIZE=0) + def test_max_page_size_disabled(self): + response = self.client.get(f'{self.url}?limit=0', format='json', **self.header) + + self.assertHttpStatus(response, status.HTTP_200_OK) + self.assertEqual(response.data['count'], 100) + self.assertIsNone(response.data['next']) + self.assertIsNone(response.data['previous']) + self.assertEqual(len(response.data['results']), 100) + + class APIDocsTestCase(TestCase): def setUp(self): diff --git a/requirements.txt b/requirements.txt index 4d156e33f..83873a55a 100644 --- a/requirements.txt +++ b/requirements.txt @@ -3,7 +3,7 @@ django-cors-headers==3.8.0 django-debug-toolbar==3.2.2 django-filter==2.4.0 django-graphiql-debug-toolbar==0.2.0 -django-mptt==0.13.2 +django-mptt==0.13.3 django-pglocks==1.0.4 django-prometheus==2.1.0 django-redis==5.0.0 @@ -18,9 +18,9 @@ gunicorn==20.1.0 Jinja2==3.0.1 Markdown==3.3.4 markdown-include==0.6.0 -mkdocs-material==7.2.5 +mkdocs-material==7.2.6 netaddr==0.8.0 -Pillow==8.3.1 +Pillow==8.3.2 psycopg2-binary==2.9.1 pycryptodome==3.10.1 PyYAML==5.4.1 diff --git a/scripts/verify-bundles.sh b/scripts/verify-bundles.sh new file mode 100755 index 000000000..e674de9bd --- /dev/null +++ b/scripts/verify-bundles.sh @@ -0,0 +1,41 @@ +#!/usr/bin/env bash + +# This script verifies the integrity of *bundled* static assets by re-running the bundling process +# and checking for changed files. Because bundle output should not change given the same source +# input, the bundle process shouldn't produce any changes. If they do, it's an indication that +# the dist files have been altered, or that dist files were not committed. In either case, tests +# should fail. + +PROJECT_STATIC="$PWD/netbox/project-static" +DIST="$PROJECT_STATIC/dist/" + +# Bundle static assets. +bundle() { + echo "Bundling static assets..." + yarn --cwd $PROJECT_STATIC bundle >/dev/null 2>&1 + if [[ $? != 0 ]]; then + echo "Error bundling static assets" + exit 1 + fi +} + +# See if any files have changed. +check_dist() { + local diff=$(git --no-pager diff $DIST) + if [[ $diff != "" ]]; then + local SHA=$(git rev-parse HEAD) + echo "Commit '$SHA' produced different static assets than were committed" + exit 1 + fi +} + +bundle +check_dist + +if [[ $? = 0 ]]; then + echo "Static asset check passed" + exit 0 +else + echo "Error checking static asset integrity" + exit 1 +fi