Merge branch 'develop' into feature

.github/workflows/ci.yml

@@ -39,13 +39,25 @@ jobs:
       with:
         node-version: ${{ matrix.node-version }}
     
+    - name: Install Yarn Package Manager
+      run: npm install -g yarn
+    
+    - name: Setup Node.js with Yarn Caching
+      uses: actions/setup-node@v2
+      with:
+        node-version: ${{ matrix.node-version }}
+        cache: yarn
+        cache-dependency-path: netbox/project-static/yarn.lock
+    
+    - name: Install Frontend Dependencies
+      run: yarn --cwd netbox/project-static
+
     - name: Install dependencies & set up configuration
       run: |
         python -m pip install --upgrade pip
         pip install -r requirements.txt
         pip install pycodestyle coverage
         ln -s configuration.testing.py netbox/netbox/configuration.py
-        yarn --cwd netbox/project-static
 
     - name: Build documentation
       run: mkdocs build
@@ -63,7 +75,7 @@ jobs:
       run: scripts/verify-bundles.sh
 
     - name: Run tests
-      run: coverage run --source="netbox/" netbox/manage.py test netbox/
+      run: coverage run --source="netbox/" netbox/manage.py test netbox/ --parallel
 
     - name: Show coverage report
       run: coverage report --skip-covered --omit *migrations*

CONTRIBUTING.md

@@ -16,13 +16,6 @@ categories for discussions:
   feature request
 * **Q&A** - Request help with installing or using NetBox
 
-### Mailing List
-
-We also have a Google Groups [mailing list](https://groups.google.com/g/netbox-discuss)
-for general discussion, however we're encouraging people to use GitHub
-discussions where possible, as it's much easier for newcomers to review past
-discussions.
-
 ### Slack
 
 For real-time chat, you can join the **#netbox** Slack channel on [NetDev Community](https://netdev.chat/).

README.md

@@ -68,7 +68,6 @@ The complete documentation for NetBox can be found at [Read the Docs](https://ne
 
 * [GitHub Discussions](https://github.com/netbox-community/netbox/discussions) - Discussion forum hosted by GitHub; ideal for Q&A and other structured discussions
 * [Slack](https://netdev.chat/) - Real-time chat hosted by the NetDev Community; best for unstructured discussion or just hanging out
-* [Google Group](https://groups.google.com/g/netbox-discuss) - Legacy mailing list; slowly being replaced by GitHub discussions
 
 ### Installation
 

docs/configuration/remote-authentication.md

@@ -35,7 +35,7 @@ The list of groups to assign a new user account when created using remote authen
 
 Default: `{}` (Empty dictionary)
 
-A mapping of permissions to assign a new user account when created using remote authentication. Each key in the dictionary should be set to a dictionary of the attributes to be applied to the permission, or `None` to allow all objects. (Requires `REMOTE_AUTH_ENABLED`.)
+A mapping of permissions to assign a new user account when created using remote authentication. Each key in the dictionary should be set to a dictionary of the attributes to be applied to the permission, or `None` to allow all objects. (Requires `REMOTE_AUTH_ENABLED` as True and `REMOTE_AUTH_GROUP_SYNC_ENABLED` as False.)
 
 ---
 
@@ -43,7 +43,7 @@ A mapping of permissions to assign a new user account when created using remote
 
 Default: `False`
 
-NetBox can be configured to support remote user authentication by inferring user authentication from an HTTP header set by the HTTP reverse proxy (e.g. nginx or Apache). Set this to `True` to enable this functionality. (Local authentication will still take effect as a fallback.)
+NetBox can be configured to support remote user authentication by inferring user authentication from an HTTP header set by the HTTP reverse proxy (e.g. nginx or Apache). Set this to `True` to enable this functionality. (Local authentication will still take effect as a fallback.) (`REMOTE_AUTH_DEFAULT_GROUPS` will not function if `REMOTE_AUTH_ENABLED` is enabled)
 
 ---
 

docs/development/index.md

@@ -7,9 +7,8 @@ NetBox is maintained as a [GitHub project](https://github.com/netbox-community/n
 There are several official forums for communication among the developers and community members:
 
 * [GitHub issues](https://github.com/netbox-community/netbox/issues) - All feature requests, bug reports, and other substantial changes to the code base **must** be documented in a GitHub issue.
-* [GitHub Discussions](https://github.com/netbox-community/netbox/discussions) - The preferred forum for general discussion and support issues. Ideal for shaping a feature request prior to submitting an issue.
+* [GitHub discussions](https://github.com/netbox-community/netbox/discussions) - The preferred forum for general discussion and support issues. Ideal for shaping a feature request prior to submitting an issue.
 * [#netbox on NetDev Community Slack](https://netdev.chat/) - Good for quick chats. Avoid any discussion that might need to be referenced later on, as the chat history is not retained long.
-* [Google Group](https://groups.google.com/g/netbox-discuss) - Legacy mailing list; slowly being phased out in favor of GitHub discussions.
 
 ## Governance
 

docs/release-notes/version-3.1.md

@@ -1,5 +1,23 @@
 # NetBox v3.1
 
+## v3.1.8 (FUTURE)
+
+### Enhancements
+
+* [#7150](https://github.com/netbox-community/netbox/issues/7150) - Linkify devices on the far side of a rack elevation
+* [#8398](https://github.com/netbox-community/netbox/issues/8398) - Embiggen configuration form fields for banner message content
+
+### Bug Fixes
+
+* [#8331](https://github.com/netbox-community/netbox/issues/8331) - Implement `replaceAll` string utility function to improve browser compatibility
+* [#8548](https://github.com/netbox-community/netbox/issues/8548) - Fix display of VC members when position is zero
+* [#8561](https://github.com/netbox-community/netbox/issues/8561) - Include option to connect a rear port to a console port
+* [#8564](https://github.com/netbox-community/netbox/issues/8564) - Fix errant table configuration key `available_columns`
+* [#8578](https://github.com/netbox-community/netbox/issues/8578) - Object change log tables should honor user's configured preferences
+* [#8604](https://github.com/netbox-community/netbox/issues/8604) - Fix tag filter on config context list filter form
+
+---
+
 ## v3.1.7 (2022-02-03)
 
 ### Enhancements

netbox/dcim/svg.py

@@ -126,10 +126,16 @@ class RackElevationSVG:
             link.add(drawing.text(str(name), insert=text, fill='white', class_='device-image-label'))
 
     def _draw_device_rear(self, drawing, device, start, end, text):
-        rect = drawing.rect(start, end, class_="slot blocked")
-        rect.set_desc(self._get_device_description(device))
-        drawing.add(rect)
-        drawing.add(drawing.text(get_device_name(device), insert=text))
+        link = drawing.add(
+            drawing.a(
+                href='{}{}'.format(self.base_url, reverse('dcim:device', kwargs={'pk': device.pk})),
+                target='_top',
+                fill='black'
+            )
+        )
+        link.set_desc(self._get_device_description(device))
+        link.add(drawing.rect(start, end, class_="slot blocked"))
+        link.add(drawing.text(get_device_name(device), insert=text))
 
         # Embed rear device type image if one exists
         if self.include_images and device.device_type.rear_image:

netbox/dcim/tables/template_code.py

@@ -311,6 +311,8 @@ REARPORT_BUTTONS = """
             </button>
             <ul class="dropdown-menu dropdown-menu-end">
                 <li><a class="dropdown-item" href="{% url 'dcim:rearport_connect' termination_a_id=record.pk termination_b_type='interface' %}?return_url={% url 'dcim:device_rearports' pk=object.pk %}">Interface</a></li>
+                <li><a class="dropdown-item" href="{% url 'dcim:rearport_connect' termination_a_id=record.pk termination_b_type='console-server-port' %}?return_url={% url 'dcim:device_rearports' pk=object.pk %}">Console Server Port</a></li>
+                <li><a class="dropdown-item" href="{% url 'dcim:rearport_connect' termination_a_id=record.pk termination_b_type='console-port' %}?return_url={% url 'dcim:device_rearports' pk=object.pk %}">Console Port</a></li>
                 <li><a class="dropdown-item" href="{% url 'dcim:rearport_connect' termination_a_id=record.pk termination_b_type='front-port' %}?return_url={% url 'dcim:device_rearports' pk=object.pk %}">Front Port</a></li>
                 <li><a class="dropdown-item" href="{% url 'dcim:rearport_connect' termination_a_id=record.pk termination_b_type='rear-port' %}?return_url={% url 'dcim:device_rearports' pk=object.pk %}">Rear Port</a></li>
                 <li><a class="dropdown-item" href="{% url 'dcim:rearport_connect' termination_a_id=record.pk termination_b_type='circuit-termination' %}?return_url={% url 'dcim:device_rearports' pk=object.pk %}">Circuit Termination</a></li>

netbox/extras/filtersets.py

@@ -330,6 +330,11 @@ class ConfigContextFilterSet(ChangeLoggedModelFilterSet):
         to_field_name='slug',
         label='Tenant (slug)',
     )
+    tag_id = django_filters.ModelMultipleChoiceFilter(
+        field_name='tags',
+        queryset=Tag.objects.all(),
+        label='Tag',
+    )
     tag = django_filters.ModelMultipleChoiceFilter(
         field_name='tags__slug',
         queryset=Tag.objects.all(),

netbox/extras/forms/filtersets.py

@@ -160,7 +160,7 @@ class TagFilterForm(FilterForm):
 
 class ConfigContextFilterForm(FilterForm):
     fieldsets = (
-        (None, ('q', 'tag')),
+        (None, ('q', 'tag_id')),
         ('Location', ('region_id', 'site_group_id', 'site_id')),
         ('Device', ('device_type_id', 'platform_id', 'role_id')),
         ('Cluster', ('cluster_type_id', 'cluster_group_id', 'cluster_id')),
@@ -222,9 +222,8 @@ class ConfigContextFilterForm(FilterForm):
         required=False,
         label=_('Tenant')
     )
-    tag = DynamicModelMultipleChoiceField(
+    tag_id = DynamicModelMultipleChoiceField(
         queryset=Tag.objects.all(),
-        to_field_name='slug',
         required=False,
         label=_('Tags')
     )

netbox/extras/tests/test_filtersets.py

@@ -12,7 +12,7 @@ from extras.filtersets import *
 from extras.models import *
 from ipam.models import IPAddress
 from tenancy.models import Tenant, TenantGroup
-from utilities.testing import BaseFilterSetTests, ChangeLoggedFilterSetTests
+from utilities.testing import BaseFilterSetTests, ChangeLoggedFilterSetTests, create_tags
 from virtualization.models import Cluster, ClusterGroup, ClusterType
 
 
@@ -444,6 +444,8 @@ class ConfigContextTestCase(TestCase, ChangeLoggedFilterSetTests):
         )
         Tenant.objects.bulk_create(tenants)
 
+        tags = create_tags('Alpha', 'Bravo', 'Charlie')
+
         for i in range(0, 3):
             is_active = bool(i % 2)
             c = ConfigContext.objects.create(
@@ -462,6 +464,7 @@ class ConfigContextTestCase(TestCase, ChangeLoggedFilterSetTests):
             c.clusters.set([clusters[i]])
             c.tenant_groups.set([tenant_groups[i]])
             c.tenants.set([tenants[i]])
+            c.tags.set([tags[i]])
 
     def test_name(self):
         params = {'name': ['Config Context 1', 'Config Context 2']}
@@ -539,13 +542,20 @@ class ConfigContextTestCase(TestCase, ChangeLoggedFilterSetTests):
         params = {'tenant_group': [tenant_groups[0].slug, tenant_groups[1].slug]}
         self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
 
-    def test_tenant_(self):
+    def test_tenant(self):
         tenants = Tenant.objects.all()[:2]
         params = {'tenant_id': [tenants[0].pk, tenants[1].pk]}
         self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
         params = {'tenant': [tenants[0].slug, tenants[1].slug]}
         self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
 
+    def test_tags(self):
+        tags = Tag.objects.all()[:2]
+        params = {'tag_id': [tags[0].pk, tags[1].pk]}
+        self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
+        params = {'tag': [tags[0].slug, tags[1].slug]}
+        self.assertEqual(self.filterset(params, self.queryset).qs.count(), 2)
+
 
 class TagTestCase(TestCase, ChangeLoggedFilterSetTests):
     queryset = Tag.objects.all()

netbox/netbox/config/parameters.py

@@ -20,19 +20,28 @@ PARAMS = (
         name='BANNER_LOGIN',
         label='Login banner',
         default='',
-        description="Additional content to display on the login page"
+        description="Additional content to display on the login page",
+        field_kwargs={
+            'widget': forms.Textarea(),
+        },
     ),
     ConfigParam(
         name='BANNER_TOP',
         label='Top banner',
         default='',
-        description="Additional content to display at the top of every page"
+        description="Additional content to display at the top of every page",
+        field_kwargs={
+            'widget': forms.Textarea(),
+        },
     ),
     ConfigParam(
         name='BANNER_BOTTOM',
         label='Bottom banner',
         default='',
-        description="Additional content to display at the bottom of every page"
+        description="Additional content to display at the bottom of every page",
+        field_kwargs={
+            'widget': forms.Textarea(),
+        },
     ),
 
     # IPAM

netbox/netbox/views/__init__.py

@@ -133,7 +133,7 @@ class HomeView(View):
         changelog = ObjectChange.objects.restrict(request.user, 'view').prefetch_related(
             'user', 'changed_object_type'
         )[:10]
-        changelog_table = ObjectChangeTable(changelog)
+        changelog_table = ObjectChangeTable(changelog, user=request.user)
 
         # Check whether a new release is available. (Only for staff/superusers.)
         new_release = None

netbox/netbox/views/generic/feature_views.py

@@ -42,7 +42,8 @@ class ObjectChangeLogView(View):
         )
         objectchanges_table = tables.ObjectChangeTable(
             data=objectchanges,
-            orderable=False
+            orderable=False,
+            user=request.user
         )
         objectchanges_table.configure(request)
 

netbox/project-static/src/select/api/apiSelect.ts

@@ -8,11 +8,12 @@ import { DynamicParamsMap } from './dynamicParams';
 import { isStaticParams, isOption } from './types';
 import {
   hasMore,
-  isTruthy,
   hasError,
-  getElement,
+  isTruthy,
   getApiData,
+  getElement,
   isApiError,
+  replaceAll,
   createElement,
   uniqueByProperty,
   findFirstAdjacent,
@@ -461,7 +462,7 @@ export class APISelect {
       // Set any primitive k/v pairs as data attributes on each option.
       for (const [k, v] of Object.entries(result)) {
         if (!['id', 'slug'].includes(k) && ['string', 'number', 'boolean'].includes(typeof v)) {
-          const key = k.replaceAll('_', '-');
+          const key = replaceAll(k, '_', '-');
           data[key] = String(v);
         }
         // Set option to disabled if the result contains a matching key and is truthy.
@@ -659,7 +660,7 @@ export class APISelect {
     for (const [key, value] of this.pathValues.entries()) {
       for (const result of this.url.matchAll(new RegExp(`({{${key}}})`, 'g'))) {
         if (isTruthy(value)) {
-          url = url.replaceAll(result[1], value.toString());
+          url = replaceAll(url, result[1], value.toString());
         }
       }
     }
@@ -741,7 +742,7 @@ export class APISelect {
    * @param id DOM ID of the other element.
    */
   private updatePathValues(id: string): void {
-    const key = id.replaceAll(/^id_/gi, '');
+    const key = replaceAll(id, /^id_/i, '');
     const element = getElement<HTMLSelectElement>(`id_${key}`);
     if (element !== null) {
       // If this element's URL contains Django template tags ({{), replace the template tag
@@ -919,16 +920,18 @@ export class APISelect {
         style.setAttribute('data-netbox', id);
 
         // Scope the CSS to apply both the list item and the selected item.
-        style.innerHTML = `
+        style.innerHTML = replaceAll(
+          `
   div.ss-values div.ss-value[data-id="${id}"],
   div.ss-list div.ss-option:not(.ss-disabled)[data-id="${id}"]
    {
     background-color: ${bg} !important;
     color: ${fg} !important;
   }
-              `
-          .replaceAll('\n', '')
-          .trim();
+              `,
+          '\n',
+          '',
+        ).trim();
 
         // Add the style element to the DOM.
         document.head.appendChild(style);

netbox/project-static/src/tableConfig.ts

@@ -11,15 +11,6 @@ function saveTableConfig(): void {
   }
 }
 
-/**
- * Delete all selected columns, which reverts the user's preferences to the default column set.
- */
-function resetTableConfig(): void {
-  for (const element of getElements<HTMLSelectElement>('select[name="columns"]')) {
-    element.value = '';
-  }
-}
-
 /**
  * Add columns to the table config select element.
  */
@@ -53,7 +44,10 @@ function removeColumns(event: Event): void {
 /**
  * Submit form configuration to the NetBox API.
  */
-async function submitFormConfig(url: string, formConfig: Dict<Dict>): Promise<APIResponse<APIUserConfig>> {
+async function submitFormConfig(
+  url: string,
+  formConfig: Dict<Dict>,
+): Promise<APIResponse<APIUserConfig>> {
   return await apiPatch<APIUserConfig>(url, formConfig);
 }
 
@@ -70,25 +64,46 @@ function handleSubmit(event: Event): void {
   const url = element.getAttribute('data-url');
   if (url == null) {
     const toast = createToast(
-        'danger',
-        'Error Updating Table Configuration',
-        'No API path defined for configuration form.'
+      'danger',
+      'Error Updating Table Configuration',
+      'No API path defined for configuration form.',
     );
     toast.show();
     return;
   }
 
+  // Determine if the form action is to reset the table config.
+  const reset = document.activeElement?.getAttribute('value') === 'Reset';
+
+  // Create an array from the dot-separated config path. E.g. tables.DevicePowerOutletTable becomes
+  // ['tables', 'DevicePowerOutletTable']
+  const path = element.getAttribute('data-config-root')?.split('.') ?? [];
+
+  if (reset) {
+    // If we're resetting the table config, create an empty object for this table. E.g.
+    // tables.PlatformTable becomes {tables: PlatformTable: {}}
+    const data = path.reduceRight<Dict<Dict>>((value, key) => ({ [key]: value }), {});
+
+    // Submit the reset for configuration to the API.
+    submitFormConfig(url, data).then(res => {
+      if (hasError(res)) {
+        const toast = createToast('danger', 'Error Resetting Table Configuration', res.error);
+        toast.show();
+      } else {
+        location.reload();
+      }
+    });
+    return;
+  }
+
   // Get all the selected options from any select element in the form.
-  const options = getSelectedOptions(element);
+  const options = getSelectedOptions(element, 'select[name=columns]');
 
   // Create an object mapping the select element's name to all selected options for that element.
   const formData: Dict<Dict<string>> = Object.assign(
     {},
     ...options.map(opt => ({ [opt.name]: opt.options })),
   );
-  // Create an array from the dot-separated config path. E.g. tables.DevicePowerOutletTable becomes
-  // ['tables', 'DevicePowerOutletTable']
-  const path = element.getAttribute('data-config-root')?.split('.') ?? [];
 
   // Create an object mapping the configuration path to the select element names, which contain the
   // selection options. E.g. {tables: {DevicePowerOutletTable: {columns: ['label', 'type']}}}
@@ -112,9 +127,6 @@ export function initTableConfig(): void {
   for (const element of getElements<HTMLButtonElement>('#save_tableconfig')) {
     element.addEventListener('click', saveTableConfig);
   }
-  for (const element of getElements<HTMLButtonElement>('#reset_tableconfig')) {
-    element.addEventListener('click', resetTableConfig);
-  }
   for (const element of getElements<HTMLButtonElement>('#add_columns')) {
     element.addEventListener('click', addColumns);
   }

netbox/project-static/src/tables/interfaceTable.ts

@@ -1,4 +1,4 @@
-import { getElements, findFirstAdjacent } from '../util';
+import { getElements, replaceAll, findFirstAdjacent } from '../util';
 
 type InterfaceState = 'enabled' | 'disabled';
 type ShowHide = 'show' | 'hide';
@@ -105,9 +105,9 @@ class ButtonState {
    */
   private toggleButton(): void {
     if (this.buttonState === 'show') {
-      this.button.innerText = this.button.innerText.replaceAll('Show', 'Hide');
+      this.button.innerText = replaceAll(this.button.innerText, 'Show', 'Hide');
     } else if (this.buttonState === 'hide') {
-      this.button.innerText = this.button.innerText.replaceAll('Hide', 'Show');
+      this.button.innerText = replaceAll(this.button.innerHTML, 'Hide', 'Show');
     }
   }
 

netbox/project-static/src/util.ts

@@ -231,11 +231,15 @@ export function scrollTo(element: Element, offset: number = 0): void {
  * Iterate through a select element's options and return an array of options that are selected.
  *
  * @param base Select element.
+ * @param selector Optionally specify a selector. 'select' by default.
  * @returns Array of selected options.
  */
-export function getSelectedOptions<E extends HTMLElement>(base: E): SelectedOption[] {
+export function getSelectedOptions<E extends HTMLElement>(
+  base: E,
+  selector: string = 'select',
+): SelectedOption[] {
   let selected = [] as SelectedOption[];
-  for (const element of base.querySelectorAll<HTMLSelectElement>('select')) {
+  for (const element of base.querySelectorAll<HTMLSelectElement>(selector)) {
     if (element !== null) {
       const select = { name: element.name, options: [] } as SelectedOption;
       for (const option of element.options) {
@@ -315,7 +319,7 @@ export function* getRowValues(table: HTMLTableRowElement): Generator<string> {
   for (const element of table.querySelectorAll<HTMLTableCellElement>('td')) {
     if (element !== null) {
       if (isTruthy(element.innerText) && element.innerText !== '—') {
-        yield element.innerText.replaceAll(/[\n\r]/g, '').trim();
+        yield replaceAll(element.innerText, '[\n\r]', '').trim();
       }
     }
   }
@@ -436,3 +440,49 @@ export function uniqueByProperty<T extends unknown, P extends keyof T>(arr: T[],
   }
   return Array.from(baseMap.values());
 }
+
+/**
+ * Replace all occurrences of a pattern with a replacement string.
+ *
+ * This is a browser-compatibility-focused drop-in replacement for `String.prototype.replaceAll()`,
+ * introduced in ES2021.
+ *
+ * @param input string to be processed.
+ * @param pattern regex pattern string or RegExp object to search for.
+ * @param replacement replacement substring with which `pattern` matches will be replaced.
+ * @returns processed version of `input`.
+ */
+export function replaceAll(input: string, pattern: string | RegExp, replacement: string): string {
+  // Ensure input is a string.
+  if (typeof input !== 'string') {
+    throw new TypeError("replaceAll 'input' argument must be a string");
+  }
+  // Ensure pattern is a string or RegExp.
+  if (typeof pattern !== 'string' && !(pattern instanceof RegExp)) {
+    throw new TypeError("replaceAll 'pattern' argument must be a string or RegExp instance");
+  }
+  // Ensure replacement is able to be stringified.
+  switch (typeof replacement) {
+    case 'boolean':
+      replacement = String(replacement);
+      break;
+    case 'number':
+      replacement = String(replacement);
+      break;
+    case 'string':
+      break;
+    default:
+      throw new TypeError("replaceAll 'replacement' argument must be stringifyable");
+  }
+
+  if (pattern instanceof RegExp) {
+    // Add global flag to existing RegExp object and deduplicate
+    const flags = Array.from(new Set([...pattern.flags.split(''), 'g'])).join('');
+    pattern = new RegExp(pattern.source, flags);
+  } else {
+    // Create a RegExp object with the global flag set.
+    pattern = new RegExp(pattern, 'g');
+  }
+
+  return input.replace(pattern, replacement);
+}

netbox/templates/dcim/device.html

@@ -33,7 +33,8 @@
                                 <a href="{{ object.site.get_absolute_url }}">{{ object.site }}</a>
                             </td>
                         </tr>
-                        <th scope="row">Location</th>
+                        <tr>
+                            <th scope="row">Location</th>
                             <td>
                             {% if object.location %}
                                 {% for location in object.location.get_ancestors %}
@@ -129,7 +130,7 @@
                                         <a href="{{ vc_member.get_absolute_url }}">{{ vc_member }}</a>
                                     </td>
                                     <td>
-                                      {% badge vc_member.vc_position %}
+                                      {% badge vc_member.vc_position show_empty=True %}
                                     </td>
                                     <td>
                                       {% if object.virtual_chassis.master == vc_member %}<i class="mdi mdi-check-bold"></i>{% endif %}