netbox/docs/plugins/development/models.md
Alexander Haase d3a3a6ba46
15692: Introduce background jobs (#16927)
* Introduce reusable BackgroundJob framework

A new abstract class can be used to implement job function classes. It
handles the necessary logic for starting and stopping jobs, including
exception handling and rescheduling of recurring jobs.

This commit also includes the migration of data source jobs to the new
framework.

* Restore using import_string for jobs

Using the 'import_string()' utility from Django allows the job script
class to be simplified, as module imports no longer need to avoid loops.
This should make it easier to queue and maintain jobs.

* Use SyncDataSourceJob for management command

Instead of maintaining two separate job execution logics, the same job
is now used for both background and interactive execution.

* Implement BackgroundJob for running scripts

The independent implementations of interactive and background script
execution have been merged into a single BackgroundJob implementation.

* Fix documentation of model features

* Ensure consitent code style

* Introduce reusable ScheduledJob

A new abstract class can be used to implement job function classes that
specialize in scheduling. These use the same logic as regular
BackgroundJobs, but ensure that they are only scheduled once at any given
time.

* Introduce reusable SystemJob

A new abstract class can be used to implement job function classes that
specialize in system background tasks (e.g. synchronization or
housekeeping). In addition to the features of the BackgroundJob and
ScheduledJob classes, these implement additional logic to not need to be
bound to an existing NetBox object and to setup job schedules on plugin
load instead of an interactive request.

* Add documentation for jobs framework

* Revert "Use SyncDataSourceJob for management"

This partially reverts commit db591d4. The 'run_now' parameter of
'enqueue()' remains, as its being used by following commits.

* Merge enqueued status into JobStatusChoices

* Fix logger for ScriptJob

* Remove job name for scripts

Because scripts are already linked through the Job Instance field, the
name is displayed twice. Removing this reduces redundancy and opens up
the possibility of simplifying the BackgroundJob framework in future
commits.

* Merge ScheduledJob into BackgroundJob

Instead of using separate classes, the logic of ScheduledJob is now
merged into the generic BackgroundJob class. This allows reusing the
same logic, but dynamically deciding whether to enqueue the same job
once or multiple times.

* Add name attribute for BackgroundJob

Instead of defining individual names on enqueue, BackgroundJob classes
can now set a job name in their meta class. This is equivalent to other
Django classes and NetBox scripts.

* Drop enqueue_sync_job() method from DataSource

* Import ScriptJob directly

* Relax requirement for Jobs to reference a specific object

* Rename 'run_now' arg on Job.enqueue() to 'immediate'

* Fix queue lookup in Job enqueue

* Collapse SystemJob into BackgroundJob

* Remove legacy JobResultStatusChoices

ChoiceSet was moved to core in 40572b5.

* Use queue 'low' for system jobs by default

System jobs usually perform low-priority background tasks and therefore
can use a different queue than 'default', which is used for regular jobs
related to specific objects.

* Add test cases for BackgroundJob handling

* Fix enqueue interval jobs

As the job's name is set by enqueue(), it must not be passed in handle()
to avoid duplicate kwargs with the same name.

* Honor schedule_at for job's enqueue_once

Not only can a job's interval change, but so can the time at which it is
scheduled to run. If a specific scheduled time is set, it will also be
checked against the current job schedule. If there are any changes, the
job is rescheduled with the new time.

* Switch BackgroundJob to regular methods

Instead of using a class method for run(), a regular method is used for
this purpose. This gives the possibility to add more convenience methods
in the future, e.g. for interacting with the job object or for logging,
as implemented for scripts.

* Fix background tasks documentation

* Test enqueue in combination with enqueue_once

* Rename background jobs to tasks (to differentiate from RQ)

* Touch up docs

* Revert "Use queue 'low' for system jobs by default"

This reverts commit b17b2050df.

* Remove system background job

This commit reverts commits 4880d81 and 0b15ecf. Using the database
'connection_created' signal for job registration feels a little wrong at
this point, as it would trigger registration very often. However, the
background job framework is prepared for this use case and can be used
by plugins once the auto-registration of jobs is solved.

* Fix runscript management command

Defining names for background jobs was disabled with fb75389. The
preceeding changes in 257976d did forget the management command.

* Use regular imports for ScriptJob

* Rename BackgroundJob to JobRunner

---------

Co-authored-by: Jeremy Stretch <jstretch@netboxlabs.com>
2024-07-30 13:31:21 -04:00

8.2 KiB

Database Models

Creating Models

If your plugin introduces a new type of object in NetBox, you'll probably want to create a Django model for it. A model is essentially a Python representation of a database table, with attributes that represent individual columns. Instances of a model (objects) can be created, manipulated, and deleted using queries. Models must be defined within a file named models.py.

Below is an example models.py file containing a model with two character (text) fields:

from django.db import models

class MyModel(models.Model):
    foo = models.CharField(max_length=50)
    bar = models.CharField(max_length=50)

    def __str__(self):
        return f'{self.foo} {self.bar}'

Every model includes by default a numeric primary key. This value is generated automatically by the database, and can be referenced as pk or id.

!!! note Model names should adhere to PEP8 standards and be CapWords (no underscores). Using underscores in model names will result in problems with permissions.

Enabling NetBox Features

Plugin models can leverage certain NetBox features by inheriting from NetBox's NetBoxModel class. This class extends the plugin model to enable features unique to NetBox, including:

  • Bookmarks
  • Change logging
  • Cloning
  • Custom fields
  • Custom links
  • Custom validation
  • Export templates
  • Journaling
  • Tags
  • Webhooks

This class performs two crucial functions:

  1. Apply any fields, methods, and/or attributes necessary to the operation of these features
  2. Register the model with NetBox as utilizing these features

Simply subclass NetBoxModel when defining a model in your plugin:

# models.py
from django.db import models
from netbox.models import NetBoxModel

class MyModel(NetBoxModel):
    foo = models.CharField()
    ...

NetBoxModel Properties

docs_url

This attribute specifies the URL at which the documentation for this model can be reached. By default, it will return /static/docs/models/<app_label>/<model_name>/. Plugin models can override this to return a custom URL. For example, you might direct the user to your plugin's documentation hosted on ReadTheDocs.

_netbox_private

By default, any model introduced by a plugin will appear in the list of available object types e.g. when creating a custom field or certain dashboard widgets. If your model is intended only for "behind the scenes use" and should not be exposed to end users, set _netbox_private to True. This will omit it from the list of general-purpose object types.

Enabling Features Individually

If you prefer instead to enable only a subset of these features for a plugin model, NetBox provides a discrete "mix-in" class for each feature. You can subclass each of these individually when defining your model. (Your model will also need to inherit from Django's built-in Model class.)

For example, if we wanted to support only tags and export templates, we would inherit from NetBox's ExportTemplatesMixin and TagsMixin classes, and from Django's Model class. (Inheriting all the available mixins is essentially the same as subclassing NetBoxModel.)

# models.py
from django.db import models
from netbox.models.features import ExportTemplatesMixin, TagsMixin

class MyModel(ExportTemplatesMixin, TagsMixin, models.Model):
    foo = models.CharField()
    ...

Database Migrations

Once you have completed defining the model(s) for your plugin, you'll need to create the database schema migrations. A migration file is essentially a set of instructions for manipulating the PostgreSQL database to support your new model, or to alter existing models. Creating migrations can usually be done automatically using Django's makemigrations management command. (Ensure that your plugin has been installed and enabled first, otherwise it won't be found.)

!!! note Enable Developer Mode NetBox enforces a safeguard around the makemigrations command to protect regular users from inadvertently creating erroneous schema migrations. To enable this command for plugin development, set DEVELOPER=True in configuration.py.

$ ./manage.py makemigrations my_plugin 
Migrations for 'my_plugin':
  /home/jstretch/animal_sounds/my_plugin/migrations/0001_initial.py
    - Create model MyModel

Next, we can apply the migration to the database with the migrate command:

$ ./manage.py migrate my_plugin
Operations to perform:
  Apply all migrations: my_plugin
Running migrations:
  Applying my_plugin.0001_initial... OK

For more information about database migrations, see the Django documentation.

Feature Mixins Reference

!!! warning Please note that only the classes which appear in this documentation are currently supported. Although other classes may be present within the features module, they are not yet supported for use by plugins.

::: netbox.models.features.BookmarksMixin

::: netbox.models.features.ChangeLoggingMixin

::: netbox.models.features.CloningMixin

::: netbox.models.features.CustomLinksMixin

::: netbox.models.features.CustomFieldsMixin

::: netbox.models.features.CustomValidationMixin

::: netbox.models.features.EventRulesMixin

!!! note EventRulesMixin was renamed from WebhooksMixin in NetBox v3.7.

::: netbox.models.features.ExportTemplatesMixin

::: netbox.models.features.JobsMixin

::: netbox.models.features.JournalingMixin

::: netbox.models.features.TagsMixin

Choice Sets

For model fields which support the selection of one or more values from a predefined list of choices, NetBox provides the ChoiceSet utility class. This can be used in place of a regular choices tuple to provide enhanced functionality, namely dynamic configuration and colorization. (See Django's documentation on the choices parameter for supported model fields.)

To define choices for a model field, subclass ChoiceSet and define a tuple named CHOICES, of which each member is a two- or three-element tuple. These elements are:

  • The database value
  • The corresponding human-friendly label
  • The assigned color (optional)

A complete example is provided below.

!!! note Authors may find it useful to declare each of the database values as constants on the class, and reference them within CHOICES members. This convention allows the values to be referenced from outside the class, however it is not strictly required.

Dynamic Configuration

Some model field choices in NetBox can be configured by an administrator. For example, the default values for the Site model's status field can be replaced or supplemented with custom choices. To enable dynamic configuration for a ChoiceSet subclass, define its key as a string specifying the model and field name to which it applies. For example:

from utilities.choices import ChoiceSet

class StatusChoices(ChoiceSet):
    key = 'MyModel.status'

To extend or replace the default values for this choice set, a NetBox administrator can then reference it under the FIELD_CHOICES configuration parameter. For example, the status field on MyModel in my_plugin would be referenced as:

FIELD_CHOICES = {
    'my_plugin.MyModel.status': (
        # Custom choices
    )
}

Example

# choices.py
from utilities.choices import ChoiceSet

class StatusChoices(ChoiceSet):
    key = 'MyModel.status'

    STATUS_FOO = 'foo'
    STATUS_BAR = 'bar'
    STATUS_BAZ = 'baz'

    CHOICES = [
        (STATUS_FOO, 'Foo', 'red'),
        (STATUS_BAR, 'Bar', 'green'),
        (STATUS_BAZ, 'Baz', 'blue'),
    ]

!!! warning For dynamic configuration to work properly, CHOICES must be a mutable list, rather than a tuple.

# models.py
from django.db import models
from .choices import StatusChoices

class MyModel(models.Model):
    status = models.CharField(
        max_length=50,
        choices=StatusChoices,
        default=StatusChoices.STATUS_FOO
    )