Closes #19829: Move object types REST API endpoint to core app

contrib/netbox-housekeeping.service

@@ -1,17 +0,0 @@
-[Unit]
-Description=NetBox Housekeeping Service
-Documentation=https://docs.netbox.dev/
-After=network-online.target
-Wants=network-online.target
-
-[Service]
-Type=simple
-
-User=netbox
-Group=netbox
-WorkingDirectory=/opt/netbox
-
-ExecStart=/opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py housekeeping
-
-[Install]
-WantedBy=multi-user.target

contrib/netbox-housekeeping.sh

@@ -1,9 +0,0 @@
-#!/bin/sh
-# This shell script invokes NetBox's housekeeping management command, which
-# intended to be run nightly. This script can be copied into your system's
-# daily cron directory (e.g. /etc/cron.daily), or referenced directly from
-# within the cron configuration file.
-#
-# If NetBox has been installed into a nonstandard location, update the paths
-# below.
-/opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py housekeeping

contrib/netbox-housekeeping.timer

@@ -1,13 +0,0 @@
-[Unit]
-Description=NetBox Housekeeping Timer
-Documentation=https://docs.netbox.dev/
-After=network-online.target
-Wants=network-online.target
-
-[Timer]
-OnCalendar=daily
-AccuracySec=1h
-Persistent=true
-
-[Install]
-WantedBy=multi-user.target

docs/administration/housekeeping.md

@@ -1,49 +0,0 @@
-# Housekeeping
-
-NetBox includes a `housekeeping` management command that should be run nightly. This command handles:
-
-* Clearing expired authentication sessions from the database
-* Deleting changelog records older than the configured [retention time](../configuration/miscellaneous.md#changelog_retention)
-* Deleting job result records older than the configured [retention time](../configuration/miscellaneous.md#job_retention)
-* Check for new NetBox releases (if [`RELEASE_CHECK_URL`](../configuration/miscellaneous.md#release_check_url) is set)
-
-This command can be invoked directly, or by using the shell script provided at `/opt/netbox/contrib/netbox-housekeeping.sh`.
-
-## Scheduling
-
-### Using Cron
-
-This script can be linked from your cron scheduler's daily jobs directory (e.g. `/etc/cron.daily`) or referenced directly within the cron configuration file.
-
-```shell
-sudo ln -s /opt/netbox/contrib/netbox-housekeeping.sh /etc/cron.daily/netbox-housekeeping
-```
-
-!!! note
-    On Debian-based systems, be sure to omit the `.sh` file extension when linking to the script from within a cron directory. Otherwise, the task may not run.
-
-### Using Systemd
-
-First, create symbolic links for the systemd service and timer files. Link the existing service and timer files from the `/opt/netbox/contrib/` directory to the `/etc/systemd/system/` directory:
-
-```bash
-sudo ln -s /opt/netbox/contrib/netbox-housekeeping.service /etc/systemd/system/netbox-housekeeping.service
-sudo ln -s /opt/netbox/contrib/netbox-housekeeping.timer /etc/systemd/system/netbox-housekeeping.timer
-```
-
-Then, reload the systemd configuration and enable the timer to start automatically at boot:
-
-```bash
-sudo systemctl daemon-reload
-sudo systemctl enable --now netbox-housekeeping.timer
-```
-
-Check the status of your timer by running:
-
-```bash
-sudo systemctl list-timers --all
-```
-
-This command will show a list of all timers, including your `netbox-housekeeping.timer`. Make sure the timer is active and properly scheduled.
-
-That's it! Your NetBox housekeeping service is now configured to run daily using systemd.

docs/installation/3-netbox.md

@@ -264,18 +264,6 @@ cd /opt/netbox/netbox
 python3 manage.py createsuperuser
 ```
 
-## Schedule the Housekeeping Task
-
-NetBox includes a `housekeeping` management command that handles some recurring cleanup tasks, such as clearing out old sessions and expired change records. Although this command may be run manually, it is recommended to configure a scheduled job using the system's `cron` daemon or a similar utility.
-
-A shell script which invokes this command is included at `contrib/netbox-housekeeping.sh`. It can be copied to or linked from your system's daily cron task directory, or included within the crontab directly. (If installing NetBox into a nonstandard path, be sure to update the system paths within this script first.)
-
-```shell
-sudo ln -s /opt/netbox/contrib/netbox-housekeeping.sh /etc/cron.daily/netbox-housekeeping
-```
-
-See the [housekeeping documentation](../administration/housekeeping.md) for further details.
-
 ## Test the Application
 
 At this point, we should be able to run NetBox's development server for testing. We can check by starting a development instance locally.

docs/installation/upgrading.md

@@ -183,13 +183,3 @@ Finally, restart the gunicorn and RQ services:
 ```no-highlight
 sudo systemctl restart netbox netbox-rq
 ```
-
-## 6. Verify Housekeeping Scheduling
-
-If upgrading from a release prior to NetBox v3.0, check that a cron task (or similar scheduled process) has been configured to run NetBox's nightly housekeeping command. A shell script which invokes this command is included at `contrib/netbox-housekeeping.sh`. It can be linked from your system's daily cron task directory, or included within the crontab directly. (If NetBox has been installed in a nonstandard path, be sure to update the system paths within this script first.)
-
-```shell
-sudo ln -s /opt/netbox/contrib/netbox-housekeeping.sh /etc/cron.daily/netbox-housekeeping
-```
-
-See the [housekeeping documentation](../administration/housekeeping.md) for further details.

docs/release-notes/version-3.0.md

@@ -434,7 +434,7 @@ A new management command has been added: `manage.py housekeeping`. This command
 * Delete change log records which have surpassed the configured retention period (if configured)
 * Check for new NetBox releases (if enabled)
 
-A convenience script for calling this command via an automated scheduler has been included at `/contrib/netbox-housekeeping.sh`. Please see the [housekeeping documentation](../administration/housekeeping.md) for further details.
+A convenience script for calling this command via an automated scheduler has been included at `/contrib/netbox-housekeeping.sh`. Please see the housekeeping documentation for further details.
 
 #### Custom Queue Support for Plugins ([#6651](https://github.com/netbox-community/netbox/issues/6651))
 

mkdocs.yml

@@ -158,7 +158,6 @@ nav:
             - Okta: 'administration/authentication/okta.md'
         - Permissions: 'administration/permissions.md'
         - Error Reporting: 'administration/error-reporting.md'
-        - Housekeeping: 'administration/housekeeping.md'
         - Replicating NetBox: 'administration/replicating-netbox.md'
         - NetBox Shell: 'administration/netbox-shell.md'
     - Data Model:

netbox/core/api/serializers.py

@@ -1,4 +1,5 @@
 from .serializers_.change_logging import *
 from .serializers_.data import *
 from .serializers_.jobs import *
+from .serializers_.object_types import *
 from .serializers_.tasks import *

netbox/core/api/urls.py

@@ -10,6 +10,7 @@ router.register('data-sources', views.DataSourceViewSet)
 router.register('data-files', views.DataFileViewSet)
 router.register('jobs', views.JobViewSet)
 router.register('object-changes', views.ObjectChangeViewSet)
+router.register('object-types', views.ObjectTypeViewSet)
 router.register('background-queues', views.BackgroundQueueViewSet, basename='rqqueue')
 router.register('background-workers', views.BackgroundWorkerViewSet, basename='rqworker')
 router.register('background-tasks', views.BackgroundTaskViewSet, basename='rqtask')

netbox/core/api/views.py

@@ -17,6 +17,7 @@ from core.utils import delete_rq_job, enqueue_rq_job, get_rq_jobs, requeue_rq_jo
 from django_rq.queues import get_redis_connection
 from django_rq.utils import get_statistics
 from django_rq.settings import QUEUES_LIST
+from netbox.api.authentication import IsAuthenticatedOrLoginNotRequired
 from netbox.api.metadata import ContentTypeMetadata
 from netbox.api.pagination import LimitOffsetListPagination
 from netbox.api.viewsets import NetBoxModelViewSet, NetBoxReadOnlyModelViewSet
@@ -85,6 +86,16 @@ class ObjectChangeViewSet(ReadOnlyModelViewSet):
     filterset_class = filtersets.ObjectChangeFilterSet
 
 
+class ObjectTypeViewSet(ReadOnlyModelViewSet):
+    """
+    Read-only list of ObjectTypes.
+    """
+    permission_classes = [IsAuthenticatedOrLoginNotRequired]
+    queryset = ObjectType.objects.order_by('app_label', 'model')
+    serializer_class = serializers.ObjectTypeSerializer
+    filterset_class = filtersets.ObjectTypeFilterSet
+
+
 class BaseRQViewSet(viewsets.ViewSet):
     """
     Base class for RQ view sets. Provides a list() method. Subclasses must implement get_data().

netbox/core/filtersets.py

@@ -1,9 +1,8 @@
+import django_filters
 from django.contrib.contenttypes.models import ContentType
 from django.db.models import Q
 from django.utils.translation import gettext as _
 
-import django_filters
-
 from netbox.filtersets import BaseFilterSet, ChangeLoggedModelFilterSet, NetBoxModelFilterSet
 from netbox.utils import get_data_backend_choices
 from users.models import User
@@ -17,6 +16,7 @@ __all__ = (
     'DataSourceFilterSet',
     'JobFilterSet',
     'ObjectChangeFilterSet',
+    'ObjectTypeFilterSet',
 )
 
 
@@ -134,6 +134,25 @@ class JobFilterSet(BaseFilterSet):
         )
 
 
+class ObjectTypeFilterSet(django_filters.FilterSet):
+    q = django_filters.CharFilter(
+        method='search',
+        label=_('Search'),
+    )
+
+    class Meta:
+        model = ObjectType
+        fields = ('id', 'app_label', 'model')
+
+    def search(self, queryset, name, value):
+        if not value.strip():
+            return queryset
+        return queryset.filter(
+            Q(app_label__icontains=value) |
+            Q(model__icontains=value)
+        )
+
+
 class ObjectChangeFilterSet(BaseFilterSet):
     q = django_filters.CharFilter(
         method='search',

netbox/core/jobs.py

@@ -1,8 +1,16 @@
 import logging
-import requests
 import sys
+from datetime import timedelta
+from importlib import import_module
 
+import requests
 from django.conf import settings
+from django.core.cache import cache
+from django.utils import timezone
+from packaging import version
+
+from core.models import Job, ObjectChange
+from netbox.config import Config
 from netbox.jobs import JobRunner, system_job
 from netbox.search.backends import search_backend
 from utilities.proxy import resolve_proxies
@@ -53,16 +61,23 @@ class SystemHousekeepingJob(JobRunner):
         if settings.DEBUG or 'test' in sys.argv:
             return
 
-        # TODO: Migrate other housekeeping functions from the `housekeeping` management command.
         self.send_census_report()
+        self.clear_expired_sessions()
+        self.prune_changelog()
+        self.delete_expired_jobs()
+        self.check_for_new_releases()
 
     @staticmethod
     def send_census_report():
         """
         Send a census report (if enabled).
         """
-        # Skip if census reporting is disabled
+        logging.info("Reporting census data...")
-        if settings.ISOLATED_DEPLOYMENT or not settings.CENSUS_REPORTING_ENABLED:
+        if settings.ISOLATED_DEPLOYMENT:
+            logging.info("ISOLATED_DEPLOYMENT is enabled; skipping")
+            return
+        if not settings.CENSUS_REPORTING_ENABLED:
+            logging.info("CENSUS_REPORTING_ENABLED is disabled; skipping")
             return
 
         census_data = {
@@ -79,3 +94,94 @@ class SystemHousekeepingJob(JobRunner):
             )
         except requests.exceptions.RequestException:
             pass
+
+    @staticmethod
+    def clear_expired_sessions():
+        """
+        Clear any expired sessions from the database.
+        """
+        logging.info("Clearing expired sessions...")
+        engine = import_module(settings.SESSION_ENGINE)
+        try:
+            engine.SessionStore.clear_expired()
+            logging.info("Sessions cleared.")
+        except NotImplementedError:
+            logging.warning(
+                f"The configured session engine ({settings.SESSION_ENGINE}) does not support "
+                f"clearing sessions; skipping."
+            )
+
+    @staticmethod
+    def prune_changelog():
+        """
+        Delete any ObjectChange records older than the configured changelog retention time (if any).
+        """
+        logging.info("Pruning old changelog entries...")
+        config = Config()
+        if not config.CHANGELOG_RETENTION:
+            logging.info("No retention period specified; skipping.")
+            return
+
+        cutoff = timezone.now() - timedelta(days=config.CHANGELOG_RETENTION)
+        logging.debug(f"Retention period: {config.CHANGELOG_RETENTION} days")
+        logging.debug(f"Cut-off time: {cutoff}")
+
+        count = ObjectChange.objects.filter(time__lt=cutoff).delete()[0]
+        logging.info(f"Deleted {count} expired records")
+
+    @staticmethod
+    def delete_expired_jobs():
+        """
+        Delete any jobs older than the configured retention period (if any).
+        """
+        logging.info("Deleting expired jobs...")
+        config = Config()
+        if not config.JOB_RETENTION:
+            logging.info("No retention period specified; skipping.")
+            return
+
+        cutoff = timezone.now() - timedelta(days=config.JOB_RETENTION)
+        logging.debug(f"Retention period: {config.CHANGELOG_RETENTION} days")
+        logging.debug(f"Cut-off time: {cutoff}")
+
+        count = Job.objects.filter(created__lt=cutoff).delete()[0]
+        logging.info(f"Deleted {count} expired records")
+
+    @staticmethod
+    def check_for_new_releases():
+        """
+        Check for new releases and cache the latest release.
+        """
+        logging.info("Checking for new releases...")
+        if settings.ISOLATED_DEPLOYMENT:
+            logging.info("ISOLATED_DEPLOYMENT is enabled; skipping")
+            return
+        if not settings.RELEASE_CHECK_URL:
+            logging.info("RELEASE_CHECK_URL is not set; skipping")
+            return
+
+        # Fetch the latest releases
+        logging.debug(f"Release check URL: {settings.RELEASE_CHECK_URL}")
+        try:
+            response = requests.get(
+                url=settings.RELEASE_CHECK_URL,
+                headers={'Accept': 'application/vnd.github.v3+json'},
+                proxies=resolve_proxies(url=settings.RELEASE_CHECK_URL)
+            )
+            response.raise_for_status()
+        except requests.exceptions.RequestException as exc:
+            logging.error(f"Error fetching release: {exc}")
+            return
+
+        # Determine the most recent stable release
+        releases = []
+        for release in response.json():
+            if 'tag_name' not in release or release.get('devrelease') or release.get('prerelease'):
+                continue
+            releases.append((version.parse(release['tag_name']), release.get('html_url')))
+        latest_release = max(releases)
+        logging.debug(f"Found {len(response.json())} releases; {len(releases)} usable")
+        logging.info(f"Latest release: {latest_release[0]}")
+
+        # Cache the most recent release
+        cache.set('latest_release', latest_release, None)

netbox/core/tests/test_api.py

@@ -7,6 +7,7 @@ from django.utils import timezone
 from rq.job import Job as RQ_Job, JobStatus
 from rq.registry import FailedJobRegistry, StartedJobRegistry
 
+from rest_framework import status
 from users.models import Token, User
 from utilities.testing import APITestCase, APIViewTestCases, TestCase
 from utilities.testing.utils import disable_logging
@@ -101,6 +102,22 @@ class DataFileTest(
         DataFile.objects.bulk_create(data_files)
 
 
+class ObjectTypeTest(APITestCase):
+
+    def test_list_objects(self):
+        object_type_count = ObjectType.objects.count()
+
+        response = self.client.get(reverse('extras-api:objecttype-list'), **self.header)
+        self.assertHttpStatus(response, status.HTTP_200_OK)
+        self.assertEqual(response.data['count'], object_type_count)
+
+    def test_get_object(self):
+        object_type = ObjectType.objects.first()
+
+        url = reverse('extras-api:objecttype-detail', kwargs={'pk': object_type.pk})
+        self.assertHttpStatus(self.client.get(url, **self.header), status.HTTP_200_OK)
+
+
 class BackgroundTaskTestCase(TestCase):
     user_permissions = ()
 

netbox/extras/api/serializers.py

@@ -1,4 +1,3 @@
-from .serializers_.objecttypes import *
 from .serializers_.attachments import *
 from .serializers_.bookmarks import *
 from .serializers_.customfields import *

netbox/extras/api/urls.py

@@ -1,5 +1,6 @@
 from django.urls import include, path
 
+from core.api.views import ObjectTypeViewSet
 from netbox.api.routers import NetBoxRouter
 from . import views
 
@@ -26,7 +27,9 @@ router.register('journal-entries', views.JournalEntryViewSet)
 router.register('config-contexts', views.ConfigContextViewSet)
 router.register('config-templates', views.ConfigTemplateViewSet)
 router.register('scripts', views.ScriptViewSet, basename='script')
-router.register('object-types', views.ObjectTypeViewSet)
+
+# TODO: Remove in NetBox v4.5
+router.register('object-types', ObjectTypeViewSet)
 
 app_name = 'extras-api'
 urlpatterns = [

netbox/extras/api/views.py

@@ -10,10 +10,9 @@ from rest_framework.mixins import ListModelMixin, RetrieveModelMixin
 from rest_framework.renderers import JSONRenderer
 from rest_framework.response import Response
 from rest_framework.routers import APIRootView
-from rest_framework.viewsets import ModelViewSet, ReadOnlyModelViewSet
+from rest_framework.viewsets import ModelViewSet
 from rq import Worker
 
-from core.models import ObjectType
 from extras import filtersets
 from extras.jobs import ScriptJob
 from extras.models import *
@@ -314,20 +313,6 @@ class ScriptViewSet(ModelViewSet):
         return Response(input_serializer.errors, status=status.HTTP_400_BAD_REQUEST)
 
 
-#
-# Object types
-#
-
-class ObjectTypeViewSet(ReadOnlyModelViewSet):
-    """
-    Read-only list of ObjectTypes.
-    """
-    permission_classes = [IsAuthenticatedOrLoginNotRequired]
-    queryset = ObjectType.objects.order_by('app_label', 'model')
-    serializer_class = serializers.ObjectTypeSerializer
-    filterset_class = filtersets.ObjectTypeFilterSet
-
-
 #
 # User dashboard
 #

netbox/extras/filtersets.py

@@ -29,7 +29,6 @@ __all__ = (
     'JournalEntryFilterSet',
     'LocalConfigContextFilterSet',
     'NotificationGroupFilterSet',
-    'ObjectTypeFilterSet',
     'SavedFilterFilterSet',
     'ScriptFilterSet',
     'TableConfigFilterSet',
@@ -788,26 +787,3 @@ class LocalConfigContextFilterSet(django_filters.FilterSet):
 
     def _local_context_data(self, queryset, name, value):
         return queryset.exclude(local_context_data__isnull=value)
-
-
-#
-# ContentTypes
-#
-
-class ObjectTypeFilterSet(django_filters.FilterSet):
-    q = django_filters.CharFilter(
-        method='search',
-        label=_('Search'),
-    )
-
-    class Meta:
-        model = ObjectType
-        fields = ('id', 'app_label', 'model')
-
-    def search(self, queryset, name, value):
-        if not value.strip():
-            return queryset
-        return queryset.filter(
-            Q(app_label__icontains=value) |
-            Q(model__icontains=value)
-        )

netbox/extras/management/commands/housekeeping.py

@@ -14,9 +14,16 @@ from utilities.proxy import resolve_proxies
 
 
 class Command(BaseCommand):
-    help = "Perform nightly housekeeping tasks. (This command can be run at any time.)"
+    help = "Perform nightly housekeeping tasks [DEPRECATED]"
 
     def handle(self, *args, **options):
+        self.stdout.write(
+            "Running this command is no longer necessary: All housekeeping tasks\n"
+            "are addressed automatically via NetBox's built-in job scheduler. It\n"
+            "will be removed in a future release.",
+            self.style.WARNING
+        )
+
         config = Config()
 
         # Clear expired authentication sessions (essentially replicating the `clearsessions` command)

netbox/extras/tests/test_api.py

@@ -3,7 +3,6 @@ import datetime
 from django.contrib.contenttypes.models import ContentType
 from django.urls import reverse
 from django.utils.timezone import make_aware, now
-from rest_framework import status
 
 from core.choices import ManagedFileRootPathChoices
 from core.events import *
@@ -921,22 +920,6 @@ class CreatedUpdatedFilterTest(APITestCase):
         self.assertEqual(response.data['results'][0]['id'], rack2.pk)
 
 
-class ObjectTypeTest(APITestCase):
-
-    def test_list_objects(self):
-        object_type_count = ObjectType.objects.count()
-
-        response = self.client.get(reverse('extras-api:objecttype-list'), **self.header)
-        self.assertHttpStatus(response, status.HTTP_200_OK)
-        self.assertEqual(response.data['count'], object_type_count)
-
-    def test_get_object(self):
-        object_type = ObjectType.objects.first()
-
-        url = reverse('extras-api:objecttype-detail', kwargs={'pk': object_type.pk})
-        self.assertHttpStatus(self.client.get(url, **self.header), status.HTTP_200_OK)
-
-
 class SubscriptionTest(APIViewTestCases.APIViewTestCase):
     model = Subscription
     brief_fields = ['display', 'id', 'object_id', 'object_type', 'url', 'user']