Commit Graph

62 Commits

Author SHA1 Message Date
Jeremy Stretch
afba5b2791 Merge branch 'main' into feature
Some checks are pending
CI / build (20.x, 3.12) (push) Waiting to run
CI / build (20.x, 3.13) (push) Waiting to run
CodeQL / Analyze (${{ matrix.language }}) (none, actions) (push) Waiting to run
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
2025-11-25 15:25:53 -05:00
Jason Novinger
82171fce7a Fixes #20638: Document bulk create support in OpenAPI schema (#20777)
Some checks are pending
CI / build (20.x, 3.10) (push) Waiting to run
CI / build (20.x, 3.11) (push) Waiting to run
CI / build (20.x, 3.12) (push) Waiting to run
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
CodeQL / Analyze (${{ matrix.language }}) (none, actions) (push) Waiting to run
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
* Fixes #20638: Document bulk create support in OpenAPI schema

POST operations on NetBoxModelViewSet endpoints accept both single
objects and arrays, but the schema only documented single objects.
This prevented API client generators from producing correct code.

Add explicit bulk_create_enabled flag to NetBoxModelViewSet and
update schema generation to emit oneOf for these endpoints.

* Address PR feedback

- Removed brittle serializer marking mechanism in favor of direct checks
  on behavior.
- Attempted to introduce a bulk_create action and then route to it on
  POST in NetBoxRouter, but ran in to several obstacles including
  breaking HTTP status code reporting in the schema. Opted to simply

* Remove unused bulk_create_enabled attr

Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>

---------

Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>
2025-11-24 09:33:39 -05:00
Jeremy Stretch
a4365be0a3 Merge branch 'main' into feature
Some checks are pending
CI / build (20.x, 3.13) (push) Waiting to run
CI / build (20.x, 3.12) (push) Waiting to run
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
CodeQL / Analyze (${{ matrix.language }}) (none, actions) (push) Waiting to run
2025-11-12 08:08:32 -05:00
Martin Hauser
87ff83ef1f feat(filtersets): Add object_type_id filter for Jobs (#20674)
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, actions) (push) Waiting to run
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
CI / build (20.x, 3.10) (push) Has been cancelled
CI / build (20.x, 3.11) (push) Has been cancelled
CI / build (20.x, 3.12) (push) Has been cancelled
Introduce a new `object_type_id` filter to enhance filtering by object
type for Jobs. Update related forms and fieldsets to incorporate the
new filter for better usability and consistency.

Fixes #20653
2025-11-04 13:58:54 -08:00
bctiemann
bcffc383bf Closes: #17936 - GFK serializer field (#20706)
Some checks failed
CI / build (20.x, 3.12) (push) Has been cancelled
CI / build (20.x, 3.13) (push) Has been cancelled
CodeQL / Analyze (${{ matrix.language }}) (none, actions) (push) Has been cancelled
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Has been cancelled
* Establish GFKSerializerField and replace get_* methods in circuits.py

* Set read_only=True

* Apply GFKSerializerField to all matching SerializerMethodFields

* Use GFKSerializerField for ObjectChangeSerializer.changed_object and EventRuleSerializer.action_object
2025-11-04 10:01:22 -05:00
Jeremy Stretch
be74436884 Closes #20304: Object owners (#20634)
Some checks failed
CI / build (20.x, 3.12) (push) Has been cancelled
CI / build (20.x, 3.13) (push) Has been cancelled
CodeQL / Analyze (${{ matrix.language }}) (none, actions) (push) Has been cancelled
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Has been cancelled
2025-10-24 13:08:01 -07:00
Jeremy Stretch
37a9d03348 Merge branch 'main' into feature 2025-10-14 13:54:47 -04:00
Martin Hauser
99e367cbaf docs(api): Correct IntegerRangeSerializer schema definition
Adjusts the schema mapping for `IntegerRangeSerializer` by setting
`match_subclasses` to `True` and refining the array definition. Adds
an example field for clarity in generated OpenAPI documentation.

Fixes #20494
2025-10-06 15:09:57 -04:00
Jeremy Stretch
23d7515b41 Merge branch 'main' into feature
Some checks failed
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Has been cancelled
CI / build (20.x, 3.12) (push) Has been cancelled
CI / build (20.x, 3.13) (push) Has been cancelled
CodeQL / Analyze (${{ matrix.language }}) (none, actions) (push) Has been cancelled
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Has been cancelled
2025-10-01 08:03:43 -04:00
Martin Hauser
9e75a2f955 fix(api): Fix schema and field definitions for OpenAPI
Add `get_internal_type()` to custom field classes for Django compatibility,
annotate path parameters and operation IDs for background endpoints, and
provide serializer context on the RQ base viewset to clear schema warnings.

Fixes #20365
2025-09-30 10:46:03 -04:00
Jeremy Stretch
c0e4d1c1e3 Closes #16137: Remove is_staff boolean from User model (#20306)
Some checks are pending
CI / build (20.x, 3.12) (push) Waiting to run
CI / build (20.x, 3.13) (push) Waiting to run
CodeQL / Analyze (${{ matrix.language }}) (none, actions) (push) Waiting to run
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
* Closes #16137: Remove is_staff boolean from User model

* Remove default is_staff value from UserManager.create_user()

* Restore staff_only on MenuItem

* Introduce IsSuperuser API permission to replace IsAdminUser

* Update and improve RQ task API view tests

* Remove is_staff attribute assignment from RemoteUserBackend
2025-09-10 16:51:59 -04:00
Jeremy Stretch
d433456e2f Merge branch 'main' into feature
Some checks are pending
CI / build (20.x, 3.10) (push) Waiting to run
CI / build (20.x, 3.11) (push) Waiting to run
CI / build (20.x, 3.12) (push) Waiting to run
CodeQL / Analyze (${{ matrix.language }}) (none, actions) (push) Waiting to run
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
2025-09-02 10:50:58 -04:00
Jeremy Stretch
0c3beec3a2 Fixes #20120: Cast RQ task args & kwargs to strings for serialization (#20166)
Some checks are pending
CI / build (20.x, 3.10) (push) Waiting to run
CI / build (20.x, 3.11) (push) Waiting to run
CI / build (20.x, 3.12) (push) Waiting to run
CodeQL / Analyze (${{ matrix.language }}) (none, python) (push) Waiting to run
CodeQL / Analyze (${{ matrix.language }}) (none, actions) (push) Waiting to run
CodeQL / Analyze (${{ matrix.language }}) (none, javascript-typescript) (push) Waiting to run
2025-08-25 07:57:35 -05:00
Jeremy Stretch
70bd0cc9e2 #19924: Expose public & features fields in API serializer and enable filtering 2025-08-14 14:40:01 -04:00
Jeremy Stretch
a8610a0e7e #19829: Update API URL for object type serializer 2025-08-14 14:40:01 -04:00
Arthur
efcf9e5b3b 20089 use get_queryset function for valid_models 2025-08-14 14:27:12 -04:00
Arthur Hanson
a585bc044e 20048 cleanup get_viewname URL resolution (#20050)
Some checks are pending
CI / build (20.x, 3.10) (push) Waiting to run
CI / build (20.x, 3.11) (push) Waiting to run
CI / build (20.x, 3.12) (push) Waiting to run
* #20048 add get_action_url utility function

* #20048 add get_action_url utility function

* #20048 add get_action_url utility function

* #20048 add get_action_url utility function

* #20048 add get_action_url utility function

* #20048 action_url template tag

* #20048 action_url template tag

* #20048 fix test

* #20048 review feedback

* #20048 fix tags
2025-08-11 08:38:19 -04:00
Jeremy Stretch
40dd36812c Merge branch 'main' into feature 2025-07-31 15:24:33 -04:00
Jeremy Stretch
24a0e1907a Closes #19713: Enable recording user messages in the change log (#19908)
Some checks are pending
CI / build (20.x, 3.10) (push) Waiting to run
CI / build (20.x, 3.11) (push) Waiting to run
CI / build (20.x, 3.12) (push) Waiting to run
* Add message field to ObjectChange model

* Set max length on changelog message

* Enable changelog messages for single object operations

* Fix tests

* Add changelog message support for bulk edit & bulk delete

* Cosmetic improvements to form fields

* Fix bulk operation templates

* Add message support for bulk import/update

* Add REST API support for changelog messages (WIP)

* Fix changelog_message assignment

* Enable changelog message support for bulk deletions

* Add documentation

* Fix changelog message support for VirtualChassis

* Add ChangeLoggingMixin to necesssary model forms

* Introduce get_random_string() utility function for tests

* Incorporate changelog messages for object view tests

* Incorporate changelog messages for object bulk view tests

* Add missing mixins for changelog message support

* Tweak test to generate expected number of change records

* Finish adding tests for changelog message functionality

* Misc cleanup

* Fixes #19956: Prevent duplicate deletion records from cascading deletions

* Tweak bulk deletion test to work around cascading deletions issue

* Correct API URL
2025-07-29 09:11:33 -05:00
Jeremy Stretch
c2d3363930 Closes #18399: Refactor logic for marking data source syncing as queued (#19960)
Some checks are pending
CI / build (20.x, 3.10) (push) Waiting to run
CI / build (20.x, 3.11) (push) Waiting to run
CI / build (20.x, 3.12) (push) Waiting to run
2025-07-28 09:04:38 -07:00
Jeremy Stretch
32fb3869a4 Closes #19829: Move object types REST API endpoint to core app
Some checks are pending
CI / build (20.x, 3.10) (push) Waiting to run
CI / build (20.x, 3.11) (push) Waiting to run
CI / build (20.x, 3.12) (push) Waiting to run
2025-07-16 14:54:03 -04:00
Jeremy Stretch
21a840c32e Closes #19816: Implement a logging mechanism for background jobs (#19838)
Some checks failed
CI / build (20.x, 3.10) (push) Has been cancelled
CI / build (20.x, 3.11) (push) Has been cancelled
CI / build (20.x, 3.12) (push) Has been cancelled
* Initial work on #19816

* Use TZ-aware timestamps

* Deserialize JobLogEntry timestamp

* Repurpose RQJobStatusColumn to display job entry level badges

* Misc cleanup

* Test logging

* Refactor HTML templates

* Update documentation
2025-07-14 08:52:50 -05:00
Jeremy Stretch
fc0acb020f Merge main into feature 2025-04-10 17:17:21 -04:00
Jeremy Stretch
77b9820577 Closes #18287: Enable periodic synchronization for data sources (#18747)
* Add sync_interval to DataSource

* Enqueue a SyncDataSourceJob when needed after saving a DataSource

* Fix logic for clearing pending jobs on interval change

* Fix lingering background tasks after modifying DataSource
2025-03-03 09:29:40 -05:00
Arthur Hanson
a24576f126 7848 Add RQ API (#17938)
* 7848 Add Background Tasks (RQ) to API

* 7848 Tasks

* 7848 cleanup

* 7848 add worker support

* 7848 switch to APIView

* 7848 Task detail view

* 7848 Task enqueue, requeue, stop

* 7848 Task enqueue, requeue, stop

* 7848 Task enqueue, requeue, stop

* 7848 tests

* 7848 tests

* 7848 OpenAPI doc generation

* 7848 OpenAPI doc generation

* 7848 review changes

* 7848 viewset

* 7848 viewset

* 7848 fix tests

* 7848 more viewsets

* 7848 fix docstring

* 7848 review comments

* 7848 review comments - get all tasks

* 7848 queue detail view

* 7848 cleanup

* 7848 cleanup

* 7848 cleanup

* 7848 cleanup

* Rename viewsets for consistency w/serializers

* Misc cleanup

* 7848 review changes

* 7848 review changes

* 7848 add test

* 7848 queue detail view

* 7848 fix tests

* 7848 fix the spectacular test failure

* 7848 fix the spectacular test failure

* Misc cleanup

---------

Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>
2024-11-26 10:01:06 -05:00
Jeremy Stretch
343a4af591 Closes #18022: Extend linter (ruff) to enforce line length limit (120 chars) (#18067)
* Enable E501 rule
* Configure ruff formatter
* Reformat migration files to fix line length violations
* Fix various E501 errors
* Move table template code to template_code.py & ignore E501 errors
* Reformat raw SQL
2024-11-21 15:58:11 -05:00
Jeremy Stretch
a8ec06687a Reomove obsolete nested serializers 2024-10-18 08:16:30 -04:00
Jeremy Stretch
7ac6dff96d Closes #17733: Replace pycodestyle with ruff (#17734)
* Resolve F541 errors

* Resolve F841 errors

* Resolve F811 errors

* Resolve F901 errors

* Resolve E714 errors

* Ignore F821 errors for GraphQL mixins

* Replace pycodestyle with ruff

* Move ignores to ruff.toml
2024-10-11 07:43:46 -04:00
Jeremy Stretch
e59f776e02 Closes #17725: Clean up import statements (#17728)
* #17725: Resolve all F401 errors

* Tweak noqa designation
2024-10-10 14:52:47 -04:00
Jeremy Stretch
6f7bf5baf4 Merge branch 'develop' into feature 2024-08-29 10:51:38 -04:00
Arthur Hanson
4f225b4e56 16670 fix OpenAPI schema generation with nested serializers (#17078)
* 16670 fix OpenAPI schema generation with nested serializers

* 16670 fix comment

* Add comment; misc cleanup

---------

Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>
2024-08-27 12:57:14 -04:00
Jeremy Stretch
b6071a80d9 Closes #16574: Add last_synced time to REST API serializer for data sources 2024-08-15 10:20:21 -04:00
Jeremy Stretch
e3d681be54 Closes #17143: Deprecate the use of dedicated nested API serializers 2024-08-15 08:00:33 -04:00
Alexander Haase
d3a3a6ba46 15692: Introduce background jobs (#16927)
* Introduce reusable BackgroundJob framework

A new abstract class can be used to implement job function classes. It
handles the necessary logic for starting and stopping jobs, including
exception handling and rescheduling of recurring jobs.

This commit also includes the migration of data source jobs to the new
framework.

* Restore using import_string for jobs

Using the 'import_string()' utility from Django allows the job script
class to be simplified, as module imports no longer need to avoid loops.
This should make it easier to queue and maintain jobs.

* Use SyncDataSourceJob for management command

Instead of maintaining two separate job execution logics, the same job
is now used for both background and interactive execution.

* Implement BackgroundJob for running scripts

The independent implementations of interactive and background script
execution have been merged into a single BackgroundJob implementation.

* Fix documentation of model features

* Ensure consitent code style

* Introduce reusable ScheduledJob

A new abstract class can be used to implement job function classes that
specialize in scheduling. These use the same logic as regular
BackgroundJobs, but ensure that they are only scheduled once at any given
time.

* Introduce reusable SystemJob

A new abstract class can be used to implement job function classes that
specialize in system background tasks (e.g. synchronization or
housekeeping). In addition to the features of the BackgroundJob and
ScheduledJob classes, these implement additional logic to not need to be
bound to an existing NetBox object and to setup job schedules on plugin
load instead of an interactive request.

* Add documentation for jobs framework

* Revert "Use SyncDataSourceJob for management"

This partially reverts commit db591d4. The 'run_now' parameter of
'enqueue()' remains, as its being used by following commits.

* Merge enqueued status into JobStatusChoices

* Fix logger for ScriptJob

* Remove job name for scripts

Because scripts are already linked through the Job Instance field, the
name is displayed twice. Removing this reduces redundancy and opens up
the possibility of simplifying the BackgroundJob framework in future
commits.

* Merge ScheduledJob into BackgroundJob

Instead of using separate classes, the logic of ScheduledJob is now
merged into the generic BackgroundJob class. This allows reusing the
same logic, but dynamically deciding whether to enqueue the same job
once or multiple times.

* Add name attribute for BackgroundJob

Instead of defining individual names on enqueue, BackgroundJob classes
can now set a job name in their meta class. This is equivalent to other
Django classes and NetBox scripts.

* Drop enqueue_sync_job() method from DataSource

* Import ScriptJob directly

* Relax requirement for Jobs to reference a specific object

* Rename 'run_now' arg on Job.enqueue() to 'immediate'

* Fix queue lookup in Job enqueue

* Collapse SystemJob into BackgroundJob

* Remove legacy JobResultStatusChoices

ChoiceSet was moved to core in 40572b5.

* Use queue 'low' for system jobs by default

System jobs usually perform low-priority background tasks and therefore
can use a different queue than 'default', which is used for regular jobs
related to specific objects.

* Add test cases for BackgroundJob handling

* Fix enqueue interval jobs

As the job's name is set by enqueue(), it must not be passed in handle()
to avoid duplicate kwargs with the same name.

* Honor schedule_at for job's enqueue_once

Not only can a job's interval change, but so can the time at which it is
scheduled to run. If a specific scheduled time is set, it will also be
checked against the current job schedule. If there are any changes, the
job is rescheduled with the new time.

* Switch BackgroundJob to regular methods

Instead of using a class method for run(), a regular method is used for
this purpose. This gives the possibility to add more convenience methods
in the future, e.g. for interacting with the job object or for logging,
as implemented for scripts.

* Fix background tasks documentation

* Test enqueue in combination with enqueue_once

* Rename background jobs to tasks (to differentiate from RQ)

* Touch up docs

* Revert "Use queue 'low' for system jobs by default"

This reverts commit b17b2050df.

* Remove system background job

This commit reverts commits 4880d81 and 0b15ecf. Using the database
'connection_created' signal for job registration feels a little wrong at
this point, as it would trigger registration very often. However, the
background job framework is prepared for this use case and can be used
by plugins once the auto-registration of jobs is solved.

* Fix runscript management command

Defining names for background jobs was disabled with fb75389. The
preceeding changes in 257976d did forget the management command.

* Use regular imports for ScriptJob

* Rename BackgroundJob to JobRunner

---------

Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>
2024-07-30 13:31:21 -04:00
Arthur Hanson
52546608f6 15156 Add display_url to REST API (#16412)
* 15156 add display_url to REST API

* 15156 fix view name

* 15156 fix typo

* 15156 fix tests

* 15156 add url display_url to base class

* 15156 add url display_url to base class

* 15156 add url display_url to base class

* 15156 review changes

* 15156 review changes

* 15156 review changes

* 15156 review changes

* 15156 remove bogus code

* 15156 remove bogus code

* 15156 review changes

* 15156 review changes

* 15156 review changes

---------

Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>
2024-06-25 09:59:58 -04:00
Jeremy Stretch
853d990c03 Closes #16388: Move change logging resources from extras to core (#16545)
* Initial work on #16388

* Misc cleanup
2024-06-17 08:03:06 -04:00
Sami Tahri
79b9ef7d0c fix: SerializedPKRelatedField schema now use nested serializer or response 2024-05-17 16:16:21 -04:00
Jeremy Stretch
46bd62fdc9 Merge branch 'develop' into feature 2024-04-22 13:23:42 -04:00
Arthur Hanson
b8cedfcc08 15582 check permissions on specific object when sync request (#15704)
* 15582 check permissions on specific object when sync request

* 15582 move permission check

* Enable translation of error message

---------

Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>
2024-04-17 10:09:50 -04:00
Jeremy Stretch
eaa3bc6576 Fixes #15570: Remove omission of M2M relations when generating writable serializers for OpenAPI schema 2024-03-28 16:17:12 -04:00
Jeremy Stretch
c382ba0ae0 Refactor REST API serializers to avoid circular imports 2024-02-27 13:26:43 -05:00
Jeremy Stretch
c146f5e1b5 Replace nested serializers with primary serializers where possible 2024-02-26 16:49:30 -05:00
Jeremy Stretch
d042e6f69d Closes #15238: Include description field in brief mode 2024-02-26 11:39:31 -05:00
Jeremy Stretch
3f3bcc5eb5 Closes #15235: Use primary serializers for REST API "brief" mode (#15246)
* Use primary serializers for brief mode

* Remove BriefModeMixin

* Correct ModuleBayTest brief_fields (see #15243)
2024-02-23 14:21:25 -05:00
Jeremy Stretch
3ddacf4b88 Merge branch 'develop' into feature 2024-02-21 16:24:23 -05:00
Abhimanyu Saharan
2e74952ac6 added missing import #15058 2024-02-16 01:20:54 +05:30
Jeremy Stretch
7abb2b2ab5 Closes #15131: Dynamic queryset annotations for REST API endpoints (#15152)
* Introduce RelatedObjectCountField

* Introduce get_annotations_for_serializer() and enable dynamic annotations

* Add RelatedObjectCountFields to serializers; remove static annotations from querysets

* Remove annotations cleanup logic from BriefModeMixin

* Annotate type for RelatedObjectCountField

* Remove redundant field on TagSerializer

* Add missing reverse relationship for power feeds to rack

* Refactor RelatedObjectCountField to take a single relationship name
2024-02-15 14:49:27 -05:00
Jeremy Stretch
b3f25a400b Closes #15087: Support for specifying a subset of API serializer fields (#15122)
* Enable dynamic field inclusion for REST API serializers

* Recurse through nested serializer when resolving prefetches

* Remove obsolete calls to prefetch_related() for API views

* Remove support for brief_prefetch_fields viewset attribute

* Rename query parameter

* Fixes #15133: Fix FHRP group representation on assignments endpoint under brief mode (#15134)

* Fixes #15133: Fix FHRP group representation on assignments endpoint under brief mode

* Update API test

* Restore get_queryset() on BriefModeMixin, minus prefetch logic

* get_prefetches_for_serializer() should reference serializer field source if set
2024-02-14 09:28:37 -05:00
Jeremy Stretch
cd8e977418 Fixes #14879: Include custom fields in REST API representation of data sources 2024-01-22 15:54:26 -05:00
Jeremy Stretch
7323668dd0 Closes #13334: Record error message on failed jobs (#14106) 2023-10-31 08:34:57 -04:00