combine topic guide with howto

Author: timgraham

docs/howto/queryable-encryption.rst

@@ -122,14 +122,103 @@ Then in your Django settings, add the custom database router to the
         "myapp.routers.EncryptedRouter",
     ]
 
+Encrypted fields
+================
+
+Now you can start using encrypted fields in your Django models.
+
+:doc:`Encrypted fields </ref/models/encrypted-fields>` may be used to protect
+sensitive data like social security numbers, credit card information, or
+personal health information. With Queryable Encryption, you can also perform
+queries on encrypted fields. To use encrypted fields in your models,
+import the necessary field types from ``django_mongodb_backend.models`` and
+define your models as usual.
+
+Here are models based on the `Python Queryable Encryption Tutorial`_::
+
+    # myapp/models.py
+    from django.db import models
+    from django_mongodb_backend.models import EmbeddedModel
+    from django_mongodb_backend.fields import (
+        EmbeddedModelField,
+        EncryptedCharField,
+        EncryptedEmbeddedModelField,
+    )
+
+
+    class PatientRecord(EmbeddedModel):
+        ssn = EncryptedCharField(max_length=11, queries={"queryType": "equality"})
+        billing = EncryptedEmbeddedModelField("Billing")
+        bill_amount = models.DecimalField(max_digits=10, decimal_places=2)
+
+    class Patient(models.Model):
+        patient_name = models.CharField(max_length=255)
+        patient_id = models.BigIntegerField()
+        patient_record = EmbeddedModelField("PatientRecord")
+
+        def __str__(self):
+            return f"{self.patient_name} ({self.patient_id})"
+
+    class Billing(EmbeddedModel):
+        cc_type = models.CharField(max_length=50)
+        cc_number = models.CharField(max_length=20)
+
+.. _Python Queryable Encryption Tutorial: https://github.com/mongodb/docs/tree/main/content/manual/manual/source/includes/qe-tutorials/python
+
+.. _qe-migrations:
+
+Migrations
+==========
+
+Once you have defined your models, create a migration as usual:
+
+.. code-block:: console
+
+    $ python manage.py makemigrations
+
+Then run the migrations with:
+
+.. code-block:: console
+
+    $ python manage.py migrate --database encrypted
+
+.. warning::
+
+    Be aware that you cannot add encrypted fields to existing models, nor can
+    you change the definition of an encrypted field, for example, to make it
+    queryable.
+
+Now create and manipulate instances of the data just like any other Django
+model data. The fields will automatically handle encryption and decryption,
+ensuring that :ref:`sensitive data is stored securely in the database
+<manual:qe-features-encryption-at-rest>`.
+
+Querying encrypted fields
+=========================
+
+In order to query encrypted fields, you must include the :ref:`queries
+<encrypted-fields-queries>` argument. For example, notice ``PatientRecord``\'s
+``ssn`` field::
+
+    class PatientRecord(EmbeddedModel):
+        ssn = EncryptedCharField(max_length=11, queries={"queryType": "equality"})
+
+You can perform a equality query just like you would on a non-encrypted field:
+
+.. code-block:: pycon
+
+    >>> patient = Patient.objects.get(patient_record__ssn="123-45-6789")
+    >>> patient.name
+    'John Doe'
+
 .. _qe-configuring-kms:
 
 Configuring the Key Management Service (KMS)
 ============================================
 
 A local KMS provider with a hardcoded key is suitable for local development and
-testing, but production environment, you should securely :ref:`store and manage your
-encryption keys <manual:qe-fundamentals-kms-providers>`.
+testing, but production environment, you should securely :ref:`store and manage
+your encryption keys <manual:qe-fundamentals-kms-providers>`.
 
 To use Queryable Encryption, you must configure a Key Management Service (KMS)
 to store and manage the encryption keys used to encrypt and decrypt data.
@@ -294,6 +383,3 @@ To configure it in your Django settings, use
     environment variable in your shell before starting your Django application::
 
         $ export DYLD_FALLBACK_LIBRARY_PATH="/path/to/mongo_crypt_shared/:$DYLD_FALLBACK_LIBRARY_PATH"
-
-You are now ready to :doc:`start developing applications
-</topics/queryable-encryption>` with Queryable Encryption!

docs/index.rst

@@ -51,7 +51,6 @@ Models
 **Topic guides:**
 
 - :doc:`topics/embedded-models`
-- :doc:`topics/queryable-encryption`
 - :doc:`topics/transactions`
 
 Forms

docs/ref/models/encrypted-fields.rst

@@ -2,15 +2,12 @@
 Encrypted fields
 ================
 
-.. versionadded:: 5.2.3
-
-Django MongoDB Backend supports :doc:`manual:core/queryable-encryption`.
+.. currentmodule:: django_mongodb_backend.fields
 
-See :doc:`/howto/queryable-encryption` for more information on how to use
-Queryable Encryption with Django MongoDB Backend.
+.. versionadded:: 5.2.3
 
-See the :doc:`/topics/queryable-encryption` topic guide for
-more information on developing applications with Queryable Encryption.
+To use encrypted fields, you must :doc:`configure Queryable Encryption
+</howto/queryable-encryption>`.
 
 The following tables detailed which fields have encrypted counterparts. In all
 cases, the encrypted field names are simply prefixed with ``Encrypted``, e.g.
@@ -48,19 +45,62 @@ cases, the encrypted field names are simply prefixed with ``Encrypted``, e.g.
 .. csv-table:: ``django_mongodb_backend.fields``
    :header: "Model Field", "Encrypted version available?"
 
-    :class:`~.fields.ArrayField`, Yes
-    :class:`~.fields.EmbeddedModelArrayField`, Yes
-    :class:`~.fields.EmbeddedModelField`, Yes
-    :class:`~.fields.ObjectIdField`, Yes
-    :class:`~.fields.PolymorphicEmbeddedModelField`, No: may be implemented in the future.
-    :class:`~.fields.PolymorphicEmbeddedModelArrayField`, No: may be implemented in the future.
+    :class:`ArrayField`, Yes
+    :class:`EmbeddedModelArrayField`, Yes
+    :class:`EmbeddedModelField`, Yes
+    :class:`ObjectIdField`, Yes
+    :class:`PolymorphicEmbeddedModelField`, No: may be implemented in the future.
+    :class:`PolymorphicEmbeddedModelArrayField`, No: may be implemented in the future.
+
+.. _encrypted-fields-queries:
+
+``EncryptedField.queries``
+--------------------------
+
+Most encrypted fields* take an optional ``queries`` argument. It's a dictionary
+that specifies the type of queries that can be performed on the field, as well
+as any query options.
 
-These fields don't support the ``queries`` argument:
+The :ref:`available query types <manual:qe-fundamentals-encrypt-query>` depend
+on your version of MongoDB. For example, in MongoDB 8.0, the supported types
+are ``equality`` and ``range``.
+
+.. admonition:: Query types vs. Django lookups
+
+    Range queries in Queryable Encryption are different from Django's
+    :ref:`range lookups <django:field-lookups>`. Range queries allow you to
+    perform comparisons on encrypted fields, while Django's range lookups are
+    used for filtering based on a range of values.
+
+\* These fields don't support the ``queries`` argument:
 
 - ``EncryptedArrayField``
 - ``EncryptedEmbeddedModelArrayField``
 - ``EncryptedEmbeddedModelField``
 
+Embedded model encryption
+=========================
+
+There are two ways to encrypt embedded models. You can either encrypt the
+entire subdocument, in which case you can't query any the subdocuments fields,
+or you can encrypt only selected fields of the subdocument.
+
+Encrypting the entire subdocument
+---------------------------------
+
+To encrypt a subdocument, use ``EncryptedEmbeddedModelField`` or
+``EncryptedEmbeddedModelArrayField``. In this case, the field's embedded model
+cannot have any encrypted fields.
+
+Encrypting selected fields of a subdocument
+-------------------------------------------
+
+To encrypt only select fields of a subdocument, use :class:`EmbeddedModelField`
+and any of the other encrypted fields on the embedded model.
+
+MongoDB doesn't support encrypting selected fields of
+``EmbeddedModelArrayField``.
+
 Limitations
 ===========
 
@@ -70,20 +110,54 @@ MongoDB imposes some restrictions on encrypted fields:
 * They cannot be part of a unique constraint.
 * They cannot be null.
 
+``QuerySet`` limitations
+------------------------
+
+In addition to :ref:`Django MongoDB Backend's QuerySet limitations
+<known-issues-limitations-querying>`, some ``QuerySet`` methods aren't
+supported on encrypted fields. Each unsupported method is followed by a sample
+error message from the database. Depending on the exact query, error messages
+may vary.
+
+- :meth:`~django.db.models.query.QuerySet.order_by`: Cannot add an encrypted
+  field as a prefix of another encrypted field.
+- :meth:`~django.db.models.query.QuerySet.alias`,
+  :meth:`~django.db.models.query.QuerySet.annotate`,
+  :meth:`~django.db.models.query.QuerySet.distinct`: Cannot group on field
+  '_id.value' which is encrypted with the random algorithm or whose encryption
+  properties are not known until runtime.
+- :meth:`~django.db.models.query.QuerySet.dates`,
+  :meth:`~django.db.models.query.QuerySet.datetimes`: If the value type is a
+  date, the type of the index must also be date (and vice versa).
+- :meth:`~django.db.models.query.QuerySet.in_bulk`: Encrypted fields can't have
+  unique constraints.
+
+# TODO: add details about joined queries after
+https://github.com/mongodb/django-mongodb-backend/pull/443 is finalized.
+
+There are also several ``QuerySet`` methods that aren't permitted on any models
+(regardless of whether or not they have encrypted fields) that use a database
+connection with Automatic Encryption. Each unsupported method is followed by a
+sample error message from the database.
+
+- :meth:`~django.db.models.query.QuerySet.update`: Multi-document updates are
+  not allowed with Queryable Encryption.
+- :meth:`~django.db.models.query.QuerySet.aggregate`,
+  :meth:`~django.db.models.query.QuerySet.count`: Aggregation stage
+  $internalFacetTeeConsumer is not allowed or supported with automatic
+  encryption.
+- :meth:`~django.db.models.query.QuerySet.union`: Aggregation stage $unionWith
+  is not allowed or supported with automatic encryption.
+
 ``EncryptedFieldMixin``
 =======================
 
 .. class:: EncryptedFieldMixin
 
     .. versionadded:: 5.2.3
 
-    A mixin that can be used to create custom encrypted fields with Queryable
-    Encryption.
-
-    To create an encrypted field, inherit from ``EncryptedFieldMixin`` and
-    your custom field class:
-
-    .. code-block:: python
+    Use this mixin to create encrypted versions of your own custom fields. For
+    example, to create an encrypted version of ``MyField``::
 
         from django.db import models
         from django_mongodb_backend.fields import EncryptedFieldMixin
@@ -93,16 +167,5 @@ MongoDB imposes some restrictions on encrypted fields:
         class MyEncryptedField(EncryptedFieldMixin, MyField):
             pass
 
-
-    You can then use your custom encrypted field in a model, specifying the
-    desired query types:
-
-    .. code-block:: python
-
-        class MyModel(models.Model):
-            my_encrypted_field = MyEncryptedField(
-                queries={"queryType": "equality"},
-            )
-            my_encrypted_field_too = MyEncryptedField(
-                queries={"queryType": "range"},
-            )
+    This adds the :ref:`queries <encrypted-fields-queries>` argument to the
+    field.

docs/topics/index.rst

@@ -9,6 +9,5 @@ know:
    :maxdepth: 2
 
    embedded-models
-   queryable-encryption
    transactions
    known-issues