From 1def2b0f1eee90681c99652cce9ab73839e41110 Mon Sep 17 00:00:00 2001 From: Mike Woofter Date: Fri, 26 Apr 2024 11:59:34 -0500 Subject: [PATCH 01/10] autobuilder From 0809a78d979fff795b969d4a783c91397a586c1c Mon Sep 17 00:00:00 2001 From: Mike Woofter <108414937+mongoKart@users.noreply.github.com> Date: Wed, 5 Feb 2025 14:25:57 -0600 Subject: [PATCH 02/10] split to multiple pages --- source/connect/mongoclient.txt | 47 +- source/databases-collections.txt | 72 +++ source/fundamentals/type-hints.txt | 435 ------------------ source/includes/type-hints/intro.rst | 6 + source/includes/type-hints/tip-more-info.rst | 5 + .../troubleshooting-client-type.rst | 13 + .../troubleshooting-incompatible-type.rst | 16 + .../type-hints/typeddict-availability.rst | 3 + source/read/retrieve.txt | 18 + source/run-command.txt | 56 +++ source/tools.txt | 24 +- source/write/bulk-write.txt | 38 ++ source/write/insert.txt | 168 +++++++ 13 files changed, 452 insertions(+), 449 deletions(-) delete mode 100644 source/fundamentals/type-hints.txt create mode 100644 source/includes/type-hints/intro.rst create mode 100644 source/includes/type-hints/tip-more-info.rst create mode 100644 source/includes/type-hints/troubleshooting-client-type.rst create mode 100644 source/includes/type-hints/troubleshooting-incompatible-type.rst create mode 100644 source/includes/type-hints/typeddict-availability.rst diff --git a/source/connect/mongoclient.txt b/source/connect/mongoclient.txt index 82d4f671..6dd39a84 100644 --- a/source/connect/mongoclient.txt +++ b/source/connect/mongoclient.txt @@ -33,6 +33,12 @@ while connected to MongoDB. This guide shows you how to create a connection string and use a ``MongoClient`` object to connect to MongoDB. +.. tip:: Type Hints + + If your application uses Python 3.5 or later, you can add type hints to your + ``MongoClient`` objects. To learn how to add type hints, see + :ref:`Type Hints `. + .. _pymongo_connection_uri: Connection URI @@ -132,11 +138,21 @@ constructor accepts. All parameters are optional. * - ``document_class`` - The default class that the client uses to decode BSON documents returned by queries. - This parameter supports the ``bson.raw_bson.RawBSONDocument`` type, as well as - subclasses of the ``collections.abc.Mapping`` type, such as ``bson.son.SON``. + This parameter supports the following types: - If you specify ``bson.son.SON`` as the document class, you must also specify types - for the key and value. + - ``bson.raw_bson.RawBSONDocument``. + - A subclass of the ``collections.abc.Mapping`` type, such as ``bson.son.SON``. + If you specify ``bson.son.SON`` as the document class, you must also specify types + for the key and value. + - A subclass of the ``TypedDict`` type. To pass a ``TypedDict`` subclass for this + parameter, you must also include the class in a type hint for your ``MongoClient`` + object, as shown in the following example: + + .. code-block:: python + + client: MongoClient[MyTypedDict] = MongoClient() + + .. include:: /includes/type-hints/typeddict-availability.rst **Data type:** ``Type[_DocumentType]`` @@ -226,28 +242,31 @@ To use multiprocessing with {+driver-short+}, write code similar to the followin Do not copy an instance of the ``MongoClient`` class from the parent process to a child process. -Type Hints +.. _pymongo-type-hints-client: + +Type Hints ---------- -If you're using Python v3.5 or later, you can add type hints to your Python code. +.. include:: /includes/type-hints/intro.rst -The following code example shows how to declare a type hint for a ``MongoClient`` -object: +To use type hints in your {+driver-short+} application, you must add a type annotation to your +``MongoClient`` object, as shown in the following example: .. code-block:: python client: MongoClient = MongoClient() -In the previous example, the code doesn't specify a type for the documents that the -``MongoClient`` object will work with. To specify a document type, -include the ``Dict[str, Any]`` type when you -create the ``MongoClient`` object, as shown in the following example: +For more accurate type information, you can include the generic document type +``Dict[str, Any]`` in your type annotation. This type matches all documents in MongoDB. +The following example shows how to add this type hint: .. code-block:: python from typing import Any, Dict client: MongoClient[Dict[str, Any]] = MongoClient() +.. include:: /includes/type-hints/tip-more-info.rst + Troubleshooting --------------- @@ -312,6 +331,10 @@ process. multithreaded contexts with ``fork()``, see `Issue 6721 `__ in the Python Issue Tracker. +.. include:: /includes/type-hints/troubleshooting-client-type.rst + +.. include:: /includes/type-hints/troubleshooting-incompatible-type.rst + API Documentation ----------------- diff --git a/source/databases-collections.txt b/source/databases-collections.txt index 0b7ee96d..93f9ec58 100644 --- a/source/databases-collections.txt +++ b/source/databases-collections.txt @@ -321,9 +321,81 @@ To learn more about supported retryable read operations, see :manual:`Retryable in the {+mdb-server+} manual. To learn more about supported retryable write operations, see :manual:`Retryable Writes ` in the {+mdb-server+} manual. +.. _pymongo-databases-collection-type-hints: + +Type Hints +---------- + +.. include:: /includes/type-hints/intro.rst + +.. include:: /includes/type-hints/tip-more-info.rst + +.. note:: TypedDict Requires Python 3.8+ + + .. include:: /includes/type-hints/typeddict-availability.rst + +Database +~~~~~~~~ + +If all documents in a database match a well-defined schema, you can specify a type hint +that uses a Python class to represent the documents' structure. By including this class +in the type hint for your ``Database`` object, you can ensure that all documents you +store or retrieve have the required structure. This provides more accurate type +checking and code completion than the default ``Dict[str, Any]`` type. + +First, define a class to represent a document from the database. The class must inherit +from the ``TypedDict`` class and must contain the same fields as the documents in the +database. After you define your class, include its name as the generic type for the +``Database`` type hint. + +The following example defines a ``Movie`` class and uses it as the +generic type for a ``Database`` type hint: + +.. code-block:: python + :emphasize-lines: 5-7, 10 + + from typing import TypedDict + from pymongo import MongoClient + from pymongo.database import Database + + class Movie(TypedDict): + name: str + year: int + + client: MongoClient = MongoClient() + database: Database[Movie] = client["test_database"] + +Collection +~~~~~~~~~~ + +Adding a generic type to a ``Collection`` type hint is similar to adding a generic type +to a ``Database`` type hint. First, define a class that inherits from the ``TypedDict`` class +and represents the structure of the +documents in the collection. Then, include the class name as the generic type for the +``Collection`` type hint, as shown in the following example: + +.. code-block:: python + :emphasize-lines: 5-7,11 + + from typing import TypedDict + from pymongo import MongoClient + from pymongo.collection import Collection + + class Movie(TypedDict): + name: str + year: int + + client: MongoClient = MongoClient() + database = client["test_database"] + collection: Collection[Movie] = database["test_collection"] + Troubleshooting --------------- +.. include:: /includes/type-hints/troubleshooting-client-type.rst + +.. include:: /includes/type-hints/troubleshooting-incompatible-type.rst + .. include:: /includes/troubleshooting/read-write-options.rst API Documentation diff --git a/source/fundamentals/type-hints.txt b/source/fundamentals/type-hints.txt deleted file mode 100644 index cb8114c8..00000000 --- a/source/fundamentals/type-hints.txt +++ /dev/null @@ -1,435 +0,0 @@ -.. _pymongo-type-hints: - -Type Hints -========== - -.. contents:: On this page - :local: - :backlinks: none - :depth: 2 - :class: singlecol - -.. facet:: - :name: genre - :values: tutorial - -.. meta:: - :keywords: mypy, type safety, compile time, type check, static - -If your IDE is configured to use `type hints `__, -it can suggest more appropriate completions and highlight errors in your code. - -All of the public APIs in PyMongo are fully type hinted, and several of them support -generic parameters for the type of document object returned when decoding BSON documents. - -For a larger set of examples that use types, see the PyMongo -`test_typing module `__. - -.. note:: - - You can also use the `Mypy `__ - tool from your command line or in continuous-integration tests. However, due to - `limitations in Mypy `__, the default - values for generic document types are not yet available. They will eventually be - ``Dict[str, any]``). - - If you are using Mypy and would like to opt out of using the provided types, add the - following lines to your - `Mypy configuration file `__: - - .. code-block:: python - - [mypy-pymongo] - follow_imports = False - -Basic Usage ------------ - -The following code example specifies the ``MongoClient`` type for the ``MongoClient`` object, but -doesn't specify a type for documents. Therefore, {+driver-short+} -uses the default, unspecified type for these documents: - -.. code-block:: python - :emphasize-lines: 2 - - >>> from pymongo import MongoClient - >>> client: MongoClient = MongoClient() - >>> collection = client.test.test - >>> inserted = collection.insert_one({"x": 1, "tags": ["dog", "cat"]}) - >>> retrieved = collection.find_one({"x": 1}) - >>> assert isinstance(retrieved, dict) - -To specify a type for documents, you can use the ``Dict[str, Any]`` type when you -create the ``MongoClient`` object, as shown in the following example: - -.. code-block:: python - :emphasize-lines: 1,3 - - >>> from typing import Any, Dict - >>> from pymongo import MongoClient - >>> client: MongoClient[Dict[str, Any]] = MongoClient() - >>> collection = client.test.test - >>> inserted = collection.insert_one({"x": 1, "tags": ["dog", "cat"]}) - >>> retrieved = collection.find_one({"x": 1}) - >>> assert isinstance(retrieved, dict) - -Typed Client ------------- - -When you create a ``MongoClient`` object, you can specify the document type used to -decode BSON documents. - -The following example shows how to specify a ``~bson.raw_bson.RawBSONDocument`` document type: - -.. code-block:: python - - >>> from pymongo import MongoClient - >>> from bson.raw_bson import RawBSONDocument - >>> client = MongoClient(document_class=RawBSONDocument) - >>> collection = client.test.test - >>> inserted = collection.insert_one({"x": 1, "tags": ["dog", "cat"]}) - >>> result = collection.find_one({"x": 1}) - >>> assert isinstance(result, RawBSONDocument) - -You can also use subclasses of ``collections.abc.Mapping`` such as ``~bson.son.SON``, -as shown in the following example: - -.. code-block:: python - - >>> from bson import SON - >>> from pymongo import MongoClient - >>> client = MongoClient(document_class=SON[str, int]) - >>> collection = client.test.test - >>> inserted = collection.insert_one({"x": 1, "y": 2}) - >>> result = collection.find_one({"x": 1}) - >>> assert result is not None - >>> assert result["x"] == 1 - -.. note:: - - When you use the ``~bson.son.SON`` document type, you must also specify types - for the key and value. The preceding example uses ``SON[str, int]``. - -Typed Collection ----------------- - -If you use a well-defined schema for the the data in a -``~pymongo.collection.Collection``, you can use the ``~typing.TypedDict`` class -to declare types for the values of the elements in the collection. - -.. important:: - - The ``TypedDict`` class is available only in Python 3.8 and later. - To use ``TypedDict`` in earlier versions of Python, install the ``typing_extensions`` - package. - -In the following example, ``Movie`` is an instance of ``TypedDict``. Each ``Movie`` object -contains two key-value pairs: ``name``, a string key with a string value, and -``year``, a string key with an integer value. - -.. code-block:: python - :emphasize-lines: 1, 4-6 - - >>> from typing import TypedDict - >>> from pymongo import MongoClient - >>> from pymongo.collection import Collection - >>> class Movie(TypedDict): - ... name: str - ... year: int - ... - >>> client: MongoClient = MongoClient() - >>> collection: Collection[Movie] = client.test.test - >>> inserted = collection.insert_one(Movie(name="Jurassic Park", year=1993)) - >>> result = collection.find_one({"name": "Jurassic Park"}) - >>> assert result is not None - >>> assert result["year"] == 1993 - -This same typing scheme works for all of the insert methods: - -- ``~pymongo.collection.Collection.insert_one()`` -- ``~pymongo.collection.Collection.insert_many()`` -- ``~pymongo.collection.Collection.bulk_write()`` - -For ``bulk_write()``, both the ``~pymongo.operations.InsertOne()`` and -``~pymongo.operations.ReplaceOne()`` operators are generic. - -The following code example shows that the results are the same as the preceding examples -when you call the ``bulk_write()`` method: - -.. code-block:: python - - >>> from typing import TypedDict - >>> from pymongo import MongoClient - >>> from pymongo.operations import InsertOne - >>> from pymongo.collection import Collection - >>> client: MongoClient = MongoClient() - >>> collection: Collection[Movie] = client.test.test - >>> inserted = collection.bulk_write([InsertOne(Movie(name="Jurassic Park", year=1993))]) - >>> result = collection.find_one({"name": "Jurassic Park"}) - >>> assert result is not None - >>> assert result["year"] == 1993 - >>> # This will raise a type-checking error, despite being present, because it is added by PyMongo. - >>> assert result["_id"] # type:ignore[typeddict-item] - -Modeling Document Types with TypedDict --------------------------------------- - -All `schema validation `__ -for inserts and updates is done on the server. These methods automatically add an ``_id`` -field to each document that doesn't include one. There are three ways to handle the ``_id`` -field in your custom ``TypedDict`` class. - -If you don't specify the ``_id`` field, {+driver-short+} automatically inserts it. -You can retrieve the value of the field at runtime, but you'll see a type error -at compile time, as shown in the following example: - -.. code-block:: python - :emphasize-lines: 12-13 - - >>> from typing import TypedDict - >>> from pymongo import MongoClient - >>> from pymongo.collection import Collection - >>> class Movie(TypedDict): - ... name: str - ... year: int - ... - >>> client: MongoClient = MongoClient() - >>> collection: Collection[Movie] = client.test.test - >>> inserted = collection.insert_one(Movie(name="Jurassic Park", year=1993)) - >>> result = collection.find_one({"name": "Jurassic Park"}) - >>> # _id is present but was added by PyMongo; this will raise a type-checking error - >>> assert result["_id"] - -You can ignore this type error by adding a ``# type:ignore`` comment at the end of -the line, as shown in the following example: - -.. code-block:: python - :emphasize-lines: 13 - - >>> from typing import TypedDict - >>> from pymongo import MongoClient - >>> from pymongo.collection import Collection - >>> class Movie(TypedDict): - ... name: str - ... year: int - ... - >>> client: MongoClient = MongoClient() - >>> collection: Collection[Movie] = client.test.test - >>> inserted = collection.insert_one(Movie(name="Jurassic Park", year=1993)) - >>> result = collection.find_one({"name": "Jurassic Park"}) - >>> # _id is present but was added by PyMongo; this will raise a type-checking error - >>> assert result["_id"] # type:ignore[typeddict-item] - -If you explicitly specify a value for the ``_id`` field, every instance of your custom -``TypedDict`` class must have a value for ``_id``. - -A third alternative is to install the ``~typing.NotRequired`` package. When you use this -package, you can access the ``_id`` field at run-time, but you won't see a type error, and -you don't need to include the field in every instance of your class. - -The following example shows how to implement these three approaches to the ``_id`` field: - -.. code-block:: python - - >>> from typing import TypedDict, NotRequired - >>> from pymongo import MongoClient - >>> from pymongo.collection import Collection - >>> from bson import ObjectId - >>> class Movie(TypedDict): - ... name: str - ... year: int - ... - >>> class ExplicitMovie(TypedDict): - ... _id: ObjectId - ... name: str - ... year: int - ... - >>> class NotRequiredMovie(TypedDict): - ... _id: NotRequired[ObjectId] - ... name: str - ... year: int - ... - >>> client: MongoClient = MongoClient() - >>> collection: Collection[Movie] = client.test.test - >>> inserted = collection.insert_one(Movie(name="Jurassic Park", year=1993)) - >>> result = collection.find_one({"name": "Jurassic Park"}) - >>> assert result is not None - >>> # This will yield a type-checking error, despite being present, because it is added by PyMongo. - >>> assert result["_id"] # type:ignore[typeddict-item] - >>> collection: Collection[ExplicitMovie] = client.test.test - >>> # Note that the _id keyword argument must be supplied - >>> inserted = collection.insert_one( - ... ExplicitMovie(_id=ObjectId(), name="Jurassic Park", year=1993) - ... ) - >>> result = collection.find_one({"name": "Jurassic Park"}) - >>> assert result is not None - >>> # This will not raise a type-checking error. - >>> assert result["_id"] - >>> collection: Collection[NotRequiredMovie] = client.test.test - >>> # Note the lack of _id, similar to the first example - >>> inserted = collection.insert_one(NotRequiredMovie(name="Jurassic Park", year=1993)) - >>> result = collection.find_one({"name": "Jurassic Park"}) - >>> assert result is not None - >>> # This will not raise a type-checking error, despite not being provided explicitly. - >>> assert result["_id"] - -.. important:: - - The ``NotRequired`` class is available only in Python 3.11 and later. - To use ``NotRequired`` in earlier versions of Python, install the ``typing_extensions`` - package. - -Typed Database --------------- - -You can also use the ``TypedDict`` class to declare that all documents in a database -match a well-defined schema, as shown in the following example: - -.. code-block:: python - :emphasize-lines: 9 - - >>> from typing import TypedDict - >>> from pymongo import MongoClient - >>> from pymongo.database import Database - >>> class Movie(TypedDict): - ... name: str - ... year: int - ... - >>> client: MongoClient = MongoClient() - >>> db: Database[Movie] = client.test - >>> collection = db.test - >>> inserted = collection.insert_one({"name": "Jurassic Park", "year": 1993}) - >>> result = collection.find_one({"name": "Jurassic Park"}) - >>> assert result is not None - >>> assert result["year"] == 1993 - -Typed Command -------------- - -When using the the ``~pymongo.database.Database.command()`` method, you can specify -the document type by providing a custom ``~bson.codec_options.CodecOptions``: - -.. code-block:: python - :emphasize-lines: 3, 5 - - >>> from pymongo import MongoClient - >>> from bson.raw_bson import RawBSONDocument - >>> from bson import CodecOptions - >>> client: MongoClient = MongoClient() - >>> options = CodecOptions(RawBSONDocument) - >>> result = client.admin.command("ping", codec_options=options) - >>> assert isinstance(result, RawBSONDocument) - -Custom ``collections.abc.Mapping`` subclasses and ``~typing.TypedDict`` -are also supported document types. Use the following syntax for ``~typing.TypedDict``: - -.. code-block:: python - - options: CodecOptions[MyTypedDict] = CodecOptions(...) - -Typed BSON Decoding -------------------- - -You can specify the document type returned by BSON-decoding functions by providing -``~bson.codec_options.CodecOptions``: - -.. code-block:: python - :emphasize-lines: 2,7,9-10 - - >>> from typing import Any, Dict - >>> from bson import CodecOptions, encode, decode - >>> class MyDict(Dict[str, Any]): - ... def foo(self): - ... return "bar" - ... - >>> options = CodecOptions(document_class=MyDict) - >>> doc = {"x": 1, "y": 2} - >>> bsonbytes = encode(doc, codec_options=options) - >>> rt_document = decode(bsonbytes, codec_options=options) - >>> assert rt_document.foo() == "bar" - -``~bson.raw_bson.RawBSONDocument`` and ``~typing.TypedDict`` are also supported document -types. Use the following syntax for ``~typing.TypedDict``: - -.. code-block:: python - - options: CodecOptions[MyTypedDict] = CodecOptions(...) - -Troubleshooting ---------------- - -Client Type Annotations -~~~~~~~~~~~~~~~~~~~~~~~ - -If you don't add a type annotation for a ``~pymongo.mongo_client.MongoClient`` object, -you might see the following Mypy error: - -.. code-block:: python - - from pymongo import MongoClient - client = MongoClient() # error: Need type annotation for "client" - -The solution is to annotate the type as -``client: MongoClient`` or ``client: MongoClient[Dict[str, Any]]``. - -Incompatible Types -~~~~~~~~~~~~~~~~~~ - -If you use the generic form of ``~pymongo.mongo_client.MongoClient``, you -might see the following Mypy error: - -.. code-block:: python - - from pymongo import MongoClient - - client: MongoClient = MongoClient() - client.test.test.insert_many( - {"a": 1} - ) # error: Dict entry 0 has incompatible type "str": "int"; - # expected "Mapping[str, Any]": "int" - -The solution is to declare the ``MongoClient`` as shown below: - -.. code-block:: python - - ``client: MongoClient[Dict[str, Any]]`` - -You might also see an ``incompatible type`` error if you pass a list to the -``insert_one()`` method, as shown in the following example: - -.. code-block:: python - - from pymongo import MongoClient - from typing import Mapping - client: MongoClient = MongoClient() - client.test.test.insert_one( - [{}] - ) # error: Argument 1 to "insert_one" of "Collection" has - # incompatible type "List[Dict[, ]]"; - # expected "Mapping[str, Any]" - -The solution is to pass a document, rather than a list, to the ``insert_one({})`` method. - -Modifying Raw BSON -~~~~~~~~~~~~~~~~~~ - -Instances of the ``~bson.raw_bson.RawBSONDocument`` class are read-only. -The following example shows the error you will see if you try to set a value on a -``RawBSONDocument`` object: - -.. code-block:: python - - from bson.raw_bson import RawBSONDocument - from pymongo import MongoClient - - client = MongoClient(document_class=RawBSONDocument) - coll = client.test.test - doc = {"my": "doc"} - coll.insert_one(doc) - retrieved = coll.find_one({"_id": doc["_id"]}) - assert retrieved is not None - assert len(retrieved.raw) > 0 - retrieved[ - "foo" - ] = "bar" # error: Unsupported target for indexed assignment - # ("RawBSONDocument") [index] diff --git a/source/includes/type-hints/intro.rst b/source/includes/type-hints/intro.rst new file mode 100644 index 00000000..605f3118 --- /dev/null +++ b/source/includes/type-hints/intro.rst @@ -0,0 +1,6 @@ +If your application uses Python 3.5 or later, you can add *type hints*, +as described in `PEP 484 `__, to your code. +Type hints denote the data types of variables, parameters, and function return +values, as well as the structure of documents. +Some IDEs can use type hints to check your code for type errors and suggest +appropriate options for code completion. \ No newline at end of file diff --git a/source/includes/type-hints/tip-more-info.rst b/source/includes/type-hints/tip-more-info.rst new file mode 100644 index 00000000..63c4b8bb --- /dev/null +++ b/source/includes/type-hints/tip-more-info.rst @@ -0,0 +1,5 @@ +See the following resources for more information about type hints: + +- `typing `__ module (Python 3.5) +- `mypy `__ +- `typing_extensions `__ package \ No newline at end of file diff --git a/source/includes/type-hints/troubleshooting-client-type.rst b/source/includes/type-hints/troubleshooting-client-type.rst new file mode 100644 index 00000000..89e05c25 --- /dev/null +++ b/source/includes/type-hints/troubleshooting-client-type.rst @@ -0,0 +1,13 @@ +Client Type Annotations +~~~~~~~~~~~~~~~~~~~~~~~ + +If you don't add a type annotation for your ``MongoClient`` object, +you might see the following mypy error: + +.. code-block:: python + + from pymongo import MongoClient + client = MongoClient() # error: Need type annotation for "client" + +The solution is to annotate the ``MongoClient`` object as +``client: MongoClient`` or ``client: MongoClient[Dict[str, Any]]``. \ No newline at end of file diff --git a/source/includes/type-hints/troubleshooting-incompatible-type.rst b/source/includes/type-hints/troubleshooting-incompatible-type.rst new file mode 100644 index 00000000..d8861aaf --- /dev/null +++ b/source/includes/type-hints/troubleshooting-incompatible-type.rst @@ -0,0 +1,16 @@ +Incompatible Type +~~~~~~~~~~~~~~~~~ + +If you use the generic form of the ``MongoClient`` class, you +might see the following mypy error: + +.. code-block:: python + + error: Dict entry 0 has incompatible type "str": "int"; + expected "Mapping[str, Any]": "int" + +The solution is to add the following type hint to your ``MongoClient`` object: + +.. code-block:: python + + ``client: MongoClient[Dict[str, Any]]`` \ No newline at end of file diff --git a/source/includes/type-hints/typeddict-availability.rst b/source/includes/type-hints/typeddict-availability.rst new file mode 100644 index 00000000..da879c5a --- /dev/null +++ b/source/includes/type-hints/typeddict-availability.rst @@ -0,0 +1,3 @@ +The ``typing`` module, which contains the ``TypedDict`` class, is available only in +Python 3.8 and later. To use the ``TypedDict`` class in earlier versions of Python, +install the ``typing_extensions`` package. \ No newline at end of file diff --git a/source/read/retrieve.txt b/source/read/retrieve.txt index 9aa955a1..907995e3 100644 --- a/source/read/retrieve.txt +++ b/source/read/retrieve.txt @@ -199,6 +199,24 @@ to the ``find()`` method: .. include:: /includes/collation-override-note.rst +Troubleshooting +--------------- + +Modifying Raw BSON +~~~~~~~~~~~~~~~~~~ + +Instances of the ``RawBSONDocument`` class are read-only. If you try to set a value on a +``RawBSONDocument`` object, you will see an error similar to one of the following: + +.. code-block:: python + + error: Unsupported target for indexed assignment + ("RawBSONDocument") [index] + +.. code-block:: python + + TypeError: 'RawBSONDocument' object does not support item assignment + .. _pymongo-retrieve-additional-information: Additional Information diff --git a/source/run-command.txt b/source/run-command.txt index 2c33550e..a743d4df 100644 --- a/source/run-command.txt +++ b/source/run-command.txt @@ -223,6 +223,62 @@ The output of this command includes information about the collections in the database, and describes the amount and size of data stored across collections. +Type Hints +---------- + +When you use the ``Database.command()`` method to run a database command, you can instruct +the method to decode the returned BSON to a specific document type. To do so, +construct a ``CodecOptions`` object and pass the name of the class the documents should +be decoded to. The class can be one of the following types: + +- ``bson.raw_bson.RawBSONDocument``. +- A subclass of the ``collections.abc.Mapping`` type, such as ``bson.son.SON``. + If you specify ``bson.son.SON`` as the document class, you must also specify types + for the key and value. +- A subclass of the ``TypedDict`` type. To pass a ``TypedDict`` subclass for this + parameter, you must also include the class in a type hint for your ``CodecOptions`` + object. + +The following example shows how to specify the document type for the result of the +``ping`` command: + +.. code-block:: python + :emphasize-lines: 3, 5 + + from pymongo import MongoClient + from bson.raw_bson import RawBSONDocument + from bson import CodecOptions + + client: MongoClient = MongoClient() + options = CodecOptions(RawBSONDocument) + result = client.admin.command("ping", codec_options=options) + +To pass a ``TypedDict`` subclass, use the following syntax: + +.. code-block:: python + + from pymongo import MongoClient + from bson.raw_bson import RawBSONDocument + from bson import CodecOptions + from typing import TypedDict + + class Movie(TypedDict): + name: str + year: int + + client: MongoClient = MongoClient() + options: CodecOptions[Movie] = CodecOptions(Movie) + result = client.admin.command("ping", codec_options=options) + +.. include:: /includes/type-hints/tip-more-info.rst + +Troubleshooting +--------------- + +.. include:: /includes/type-hints/troubleshooting-client-type.rst + +.. include:: /includes/type-hints/troubleshooting-incompatible-type.rst + .. _pymongo-addtl-info-runcommand: Additional Information diff --git a/source/tools.txt b/source/tools.txt index bf70301a..d3516d83 100644 --- a/source/tools.txt +++ b/source/tools.txt @@ -260,6 +260,27 @@ daemon in the global application group: and in the `Multiple Python Sub Interpreters `__ section of the mod_wsgi documentation. +Type Checkers +------------- + +For a list of tools that can use type hints to detect errors in your code, +see `Static Typing with Python `__ +in the ``typing`` module documentation. + +.. note:: + + The default values for generic document types are not yet available in Mypy. + For a discussion of the Mypy limitations that caused this issue, see the + :github:`Mypy GitHub repository `. + + If you're using Mypy and want to opt out of using the provided types, add the + following lines to your Mypy configuration file: + + .. code-block:: python + + [mypy-pymongo] + follow_imports = False + Alternative Python Drivers -------------------------- @@ -280,5 +301,4 @@ This section lists alternatives to {+driver-short+}. `PythonAnywhere `__ does not support. For more information, see - the relevant `Jira ticket. `__ - + the relevant `Jira ticket. `__ \ No newline at end of file diff --git a/source/write/bulk-write.txt b/source/write/bulk-write.txt index ea2dd73d..fc97bdd3 100644 --- a/source/write/bulk-write.txt +++ b/source/write/bulk-write.txt @@ -517,6 +517,44 @@ The ``MongoClient.bulk_write()`` method returns a ``ClientBulkWriteResult`` obje * - ``upserted_count`` - | The number of documents upserted, if any. +Type Hints +---------- + +.. include:: /includes/type-hints/intro.rst + +When you use the ``bulk_write()`` method to perform an ``InsertOne`` or ``ReplaceOne`` +operation, +The following code example shows how to insert +- ``~pymongo.collection.Collection.bulk_write()`` + +For ``bulk_write()``, both the ``~pymongo.operations.InsertOne()`` and +``~pymongo.operations.ReplaceOne()`` operators are generic. + +The following code example shows that the results are the same as the preceding examples +when you call the ``bulk_write()`` method: + +.. code-block:: python + + from typing import TypedDict + from pymongo import MongoClient + from pymongo.operations import InsertOne + from pymongo.collection import Collection + client: MongoClient = MongoClient() + collection: Collection[Movie] = client.test.test + inserted = collection.bulk_write([InsertOne(Movie(name="Jurassic Park", year=1993))]) + result = collection.find_one({"name": "Jurassic Park"}) + assert result is not None + assert result["year"] == 1993 + +.. include:: /includes/type-hints/tip-more-info.rst + +Troubleshooting +--------------- + +.. include:: /includes/type-hints/troubleshooting-client-type.rst + +.. include:: /includes/type-hints/troubleshooting-incompatible-type.rst + Additional Information ---------------------- diff --git a/source/write/insert.txt b/source/write/insert.txt index 19ed2b42..35ce687d 100644 --- a/source/write/insert.txt +++ b/source/write/insert.txt @@ -140,6 +140,174 @@ document-level validation. sample_restaurants.restaurants.insert_many(document_list, bypass_document_validation = True) +Type Hints +---------- + +.. include:: /includes/type-hints/intro.rst + +When you call the ``insert_one()`` or ``insert_many()`` method, you can pass one or more +instances of a custom class that represents the documents in the collection. + +To create +a custom type, define a class that inherits from the ``TypedDict`` class. In the class, +add a property for each field in the document. + +After you define the class, you can insert instances of the class. The following example +defines a ``Movie`` class to represent documents from the +``sample_mflix.movies`` collection. +Each ``Movie`` object contains two key-value pairs: ``name``, a string key with a string +value, and ``year``, a string key with an integer value. +The code then uses the ``insert_one()`` method to insert a ``Movie`` object. + +.. code-block:: python + :emphasize-lines: 10 + + from typing import TypedDict + + class Movie(TypedDict): + name: str + year: int + + client: MongoClient = MongoClient() + database = client["test_database"] + collection: Collection[Movie] = database["test_collection"] + inserted = collection.insert_one(Movie(name="Jurassic Park", year=1993)) + +.. include:: /includes/type-hints/tip-more-info.rst + +Troubleshooting +--------------- + +.. include:: /includes/type-hints/troubleshooting-client-type.rst + +.. include:: /includes/type-hints/troubleshooting-incompatible-type.rst + +You might see a similar error if you pass a list to the ``insert_one()`` method: + +.. code-block:: bash + + error: Argument 1 to "insert_one" of "Collection" has + incompatible type "List[Dict[, ]]"; + expected "Mapping[str, Any]" + +This error occurs because the ``insert_one()`` method accepts a document, not a list. +You can resolve this error by passing a document to the ``insert_one()`` method or by +calling the ``insert_many()`` method instead. + +TypedDict Missing _id Key +~~~~~~~~~~~~~~~~~~~~~~~~~ + +As discussed above, if you don't specify the ``_id`` field, {+driver-short+} automatically +inserts it into the document. +You can retrieve the value of the ``_id`` field at runtime, but if you use MyPy or another +tool to perform static type-checking, it won't find the ``_id`` field in your class and +will show an error similar to the following: + +.. code-block:: bash + + TypedDict has no key "_id" + +This is caused by code similar to the following: + +.. code-block:: python + :emphasize-lines: 13 + + from typing import TypedDict + from pymongo import MongoClient + from pymongo.collection import Collection + class Movie(TypedDict): + name: str + year: int + + client: MongoClient = MongoClient() + collection: Collection[Movie] = client.test.test + inserted = collection.insert_one(Movie(name="Jurassic Park", year=1993)) + result = collection.find_one({"name": "Jurassic Park"}) + # _id is present but was added by PyMongo; this will raise a type-checking error + assert result["_id"] + +One solution is to add a ``# type:ignore`` +comment to the end of the line that uses the ``_id`` field. This comment instructs the +type-checking tool to ignore any errors that the line causes. The following example shows +how to implement this solution; + +.. code-block:: python + :emphasize-lines: 15 + + from typing import TypedDict + from pymongo import MongoClient + from pymongo.collection import Collection + + class Movie(TypedDict): + name: str + year: int + + collection: Collection[Movie] = client.test.test + inserted = collection.insert_one( + Movie(name="Jurassic Park", year=1993) + ) + result = collection.find_one({"name": "Jurassic Park"}) + assert result is not None + assert result["_id"] # type:ignore[typeddict-item] + +Instead of ignoring the type error, you can avoid it by including the ``_id`` field in +your model class, and explicitly specifying a value for this field when you create the +class instance. The following code shows how to implement this solution: + +.. code-block:: python + :emphasize-lines: 4,7,13 + + from typing import TypedDict + from pymongo import MongoClient + from pymongo.collection import Collection + from bson import ObjectId + + class Movie(TypedDict): + _id: ObjectId + name: str + year: int + + collection: Collection[ExplicitMovie] = client.test.test + inserted = collection.insert_one( + ExplicitMovie(_id=ObjectId(), name="Jurassic Park", year=1993) + ) + result = collection.find_one({"name": "Jurassic Park"}) + assert result is not None + assert result["_id"] + +One drawback to adding the ``_id`` field to your custom class is that you must include a +value for the field for every instance of the class that you create. +To avoid this, you can install the +``typing.NotRequired`` package, which includes the ``NotRequired`` type hint. If you +use this type hint for the ``_id`` field, you can access the value of the ``_id`` field +at runtime without seeing any compile-time type errors. + +The following code example shows how to implement this solution: + +.. code-block:: python + + from typing import TypedDict, NotRequired + from pymongo import MongoClient + from pymongo.collection import Collection + from bson import ObjectId + + class Movie(TypedDict): + _id: NotRequired[ObjectId] + name: str + year: int + + client: MongoClient = MongoClient() + collection: Collection[Movie] = client.test.test + inserted = collection.insert_one(Movie(name="Jurassic Park", year=1993)) + result = collection.find_one({"name": "Jurassic Park"}) + assert result is not None + assert result["_id"] + +.. important:: NotRequired Requires Python 3.11+ + + The ``NotRequired`` class is available only in Python 3.11 and later. + To use ``NotRequired`` in earlier versions of Python, install the ``typing_extensions`` + package instead. Additional Information ---------------------- From 8f7ba00f87458d379b76500abec9961973a81786 Mon Sep 17 00:00:00 2001 From: Mike Woofter <108414937+mongoKart@users.noreply.github.com> Date: Thu, 6 Feb 2025 13:48:28 -0600 Subject: [PATCH 03/10] fixes --- source/connect/mongoclient.txt | 18 ++--- source/databases-collections.txt | 4 +- source/includes/type-hints/tip-more-info.rst | 10 ++- .../troubleshooting-client-type.rst | 2 +- .../troubleshooting-incompatible-type.rst | 4 +- .../type-hints/typeddict-availability.rst | 6 +- source/includes/write/bulk-write.py | 24 ++++++ source/run-command.txt | 21 +++--- source/write/bulk-write.txt | 30 ++++++++ source/write/insert.txt | 75 ++++++++++--------- 10 files changed, 123 insertions(+), 71 deletions(-) diff --git a/source/connect/mongoclient.txt b/source/connect/mongoclient.txt index 6dd39a84..aa5d9498 100644 --- a/source/connect/mongoclient.txt +++ b/source/connect/mongoclient.txt @@ -33,12 +33,6 @@ while connected to MongoDB. This guide shows you how to create a connection string and use a ``MongoClient`` object to connect to MongoDB. -.. tip:: Type Hints - - If your application uses Python 3.5 or later, you can add type hints to your - ``MongoClient`` objects. To learn how to add type hints, see - :ref:`Type Hints `. - .. _pymongo_connection_uri: Connection URI @@ -125,7 +119,7 @@ constructor accepts. All parameters are optional. :wikipedia:`round-robin DNS ` addresses. **Data type:** ``Union[str, Sequence[str]]`` - **Default value:** ``"localhost"`` + | **Default value:** ``"localhost"`` * - ``port`` - The port number {+mdb-server+} is running on. @@ -134,7 +128,7 @@ constructor accepts. All parameters are optional. instead of using this parameter. **Data type:** ``int`` - **Default value:** ``27017`` + | **Default value:** ``27017`` * - ``document_class`` - The default class that the client uses to decode BSON documents returned by queries. @@ -257,16 +251,14 @@ To use type hints in your {+driver-short+} application, you must add a type anno client: MongoClient = MongoClient() For more accurate type information, you can include the generic document type -``Dict[str, Any]`` in your type annotation. This type matches all documents in MongoDB. -The following example shows how to add this type hint: +``Dict[str, Any]`` in your type annotation. This data type matches all documents in MongoDB. +The following example shows how to include this data type in your type annotation: .. code-block:: python from typing import Any, Dict client: MongoClient[Dict[str, Any]] = MongoClient() -.. include:: /includes/type-hints/tip-more-info.rst - Troubleshooting --------------- @@ -341,4 +333,4 @@ API Documentation To learn more about creating a ``MongoClient`` object in {+driver-short+}, see the following API documentation: -- `MongoClient <{+api-root+}pymongo/mongo_client.html#pymongo.mongo_client.MongoClient>`__ \ No newline at end of file +- `MongoClient <{+api-root+}pymongo/mongo_client.html#pymongo.mongo_client.MongoClient>`__ \ No newline at end of file diff --git a/source/databases-collections.txt b/source/databases-collections.txt index 93f9ec58..f50ca2cf 100644 --- a/source/databases-collections.txt +++ b/source/databases-collections.txt @@ -328,9 +328,7 @@ Type Hints .. include:: /includes/type-hints/intro.rst -.. include:: /includes/type-hints/tip-more-info.rst - -.. note:: TypedDict Requires Python 3.8+ +.. note:: TypedDict in Python 3.7 and Earlier .. include:: /includes/type-hints/typeddict-availability.rst diff --git a/source/includes/type-hints/tip-more-info.rst b/source/includes/type-hints/tip-more-info.rst index 63c4b8bb..64482e8e 100644 --- a/source/includes/type-hints/tip-more-info.rst +++ b/source/includes/type-hints/tip-more-info.rst @@ -1,5 +1,7 @@ -See the following resources for more information about type hints: +.. tip:: Learn More About Type Hints -- `typing `__ module (Python 3.5) -- `mypy `__ -- `typing_extensions `__ package \ No newline at end of file + See the following resources for more information about type hints: + + - `typing `__ module (Python 3.5+) + - `mypy `__ + - `typing_extensions `__ package \ No newline at end of file diff --git a/source/includes/type-hints/troubleshooting-client-type.rst b/source/includes/type-hints/troubleshooting-client-type.rst index 89e05c25..85e9c2e1 100644 --- a/source/includes/type-hints/troubleshooting-client-type.rst +++ b/source/includes/type-hints/troubleshooting-client-type.rst @@ -2,7 +2,7 @@ Client Type Annotations ~~~~~~~~~~~~~~~~~~~~~~~ If you don't add a type annotation for your ``MongoClient`` object, -you might see the following mypy error: +your type checker might show an error similar to the following: .. code-block:: python diff --git a/source/includes/type-hints/troubleshooting-incompatible-type.rst b/source/includes/type-hints/troubleshooting-incompatible-type.rst index d8861aaf..4abb01bc 100644 --- a/source/includes/type-hints/troubleshooting-incompatible-type.rst +++ b/source/includes/type-hints/troubleshooting-incompatible-type.rst @@ -1,8 +1,8 @@ Incompatible Type ~~~~~~~~~~~~~~~~~ -If you use the generic form of the ``MongoClient`` class, you -might see the following mypy error: +If you use the generic form of the ``MongoClient`` class, your type checker might +show an error similar to the following: .. code-block:: python diff --git a/source/includes/type-hints/typeddict-availability.rst b/source/includes/type-hints/typeddict-availability.rst index da879c5a..98c34411 100644 --- a/source/includes/type-hints/typeddict-availability.rst +++ b/source/includes/type-hints/typeddict-availability.rst @@ -1,3 +1,3 @@ -The ``typing`` module, which contains the ``TypedDict`` class, is available only in -Python 3.8 and later. To use the ``TypedDict`` class in earlier versions of Python, -install the ``typing_extensions`` package. \ No newline at end of file +The examples in this section use the ``TypedDict`` class from the ``typing`` module. This +module is available only in Python 3.8 and later. To use the ``TypedDict`` class in +earlier versions of Python, install the ``typing_extensions`` package. \ No newline at end of file diff --git a/source/includes/write/bulk-write.py b/source/includes/write/bulk-write.py index 82793cf7..1c6c36f5 100644 --- a/source/includes/write/bulk-write.py +++ b/source/includes/write/bulk-write.py @@ -10,6 +10,17 @@ ) # end-bulk-insert-one +# start-bulk-insert-one-typed +class Restaurant (TypedDict): + name: str + cuisine: str + borough: str + restaurant_id: str + +operation = pymongo.InsertOne(Restaurant( + name="Mongo's Deli", cuisine="Sandwiches", borough="Manhattan", restaurant_id="1234")) +# end-bulk-insert-one-typed + # start-bulk-update-one operation = UpdateOne( namespace="sample_restaurants.restaurants", @@ -39,6 +50,19 @@ ) # end-bulk-replace-one +# start-bulk-replace-one-typed +class Restaurant (TypedDict): + name: str + cuisine: str + borough: str + restaurant_id: str + +operation = pymongo.ReplaceOne( + { "restaurant_id": "1234" }, + Restaurant(name="Mongo's Pizza", cuisine="Pizza", borough="Brooklyn", restaurant_id="5678") +) +# end-bulk-replace-one-typed + # start-bulk-delete-one operation = DeleteOne( namespace="sample_restaurants.restaurants", diff --git a/source/run-command.txt b/source/run-command.txt index a743d4df..59ae9e38 100644 --- a/source/run-command.txt +++ b/source/run-command.txt @@ -226,10 +226,9 @@ collections. Type Hints ---------- -When you use the ``Database.command()`` method to run a database command, you can instruct -the method to decode the returned BSON to a specific document type. To do so, -construct a ``CodecOptions`` object and pass the name of the class the documents should -be decoded to. The class can be one of the following types: +The ``Database.command()`` method can decode the returned BSON documents to instances +of a specific class. To specify this class, construct a ``CodecOptions`` object and pass +the class name. The class can be one of the following types: - ``bson.raw_bson.RawBSONDocument``. - A subclass of the ``collections.abc.Mapping`` type, such as ``bson.son.SON``. @@ -239,11 +238,13 @@ be decoded to. The class can be one of the following types: parameter, you must also include the class in a type hint for your ``CodecOptions`` object. -The following example shows how to specify the document type for the result of the -``ping`` command: +.. include:: /includes/type-hints/typeddict-availability.rst + +The following example decodes the BSON returned by the ``ping`` command to instances +of the ``RawBSONDocument`` class: .. code-block:: python - :emphasize-lines: 3, 5 + :emphasize-lines: 3, 6-7 from pymongo import MongoClient from bson.raw_bson import RawBSONDocument @@ -253,9 +254,11 @@ The following example shows how to specify the document type for the result of t options = CodecOptions(RawBSONDocument) result = client.admin.command("ping", codec_options=options) -To pass a ``TypedDict`` subclass, use the following syntax: +To decode BSON to a subclass of the ``TypedDict`` class, specify the class name in +the ``CodecOptions`` type hint, as shown in the following example: .. code-block:: python + :emphasize-lines: 4, 6-8, 11 from pymongo import MongoClient from bson.raw_bson import RawBSONDocument @@ -270,8 +273,6 @@ To pass a ``TypedDict`` subclass, use the following syntax: options: CodecOptions[Movie] = CodecOptions(Movie) result = client.admin.command("ping", codec_options=options) -.. include:: /includes/type-hints/tip-more-info.rst - Troubleshooting --------------- diff --git a/source/write/bulk-write.txt b/source/write/bulk-write.txt index fc97bdd3..61cd234f 100644 --- a/source/write/bulk-write.txt +++ b/source/write/bulk-write.txt @@ -95,6 +95,21 @@ The following example creates an instance of ``InsertOne``: :language: python :copyable: +You can also create an instance of ``InsertOne`` by passing an instance of a custom class +to the constructor. This provides additional type safety if you're using a type-checking +tool. The instance you pass must inherit from the ``TypedDict`` class. + +.. include:: /includes/type-hints/typeddict-availability.rst + +The following example constructs an ``InsertOne`` instance by using a custom +class for added type safety: + +.. literalinclude:: /includes/write/bulk-write.py + :start-after: start-bulk-insert-one-typed + :end-before: end-bulk-insert-one-typed + :language: python + :copyable: + To insert multiple documents, create an instance of ``InsertOne`` for each document. .. include:: /includes/write/unique-id-note.rst @@ -157,6 +172,21 @@ The following example creates an instance of ``ReplaceOne``: :language: python :copyable: +You can also create an instance of ``ReplaceOne`` by passing an instance of a custom class +to the constructor. This provides additional type safety if you're using a type-checking +tool. The instance you pass must inherit from the ``TypedDict`` class. + +.. include:: /includes/type-hints/typeddict-availability.rst + +The following example constructs a ``ReplaceOne`` instance by using a custom +class for added type safety: + +.. literalinclude:: /includes/write/bulk-write.py + :start-after: start-bulk-replace-one-typed + :end-before: end-bulk-replace-one-typed + :language: python + :copyable: + To replace multiple documents, you must create an instance of ``ReplaceOne`` for each document. Delete Operations diff --git a/source/write/insert.txt b/source/write/insert.txt index 35ce687d..2790b244 100644 --- a/source/write/insert.txt +++ b/source/write/insert.txt @@ -60,6 +60,23 @@ The following example inserts a document into the ``restaurants`` collection: sample_restaurants.restaurants.insert_one({"name" : "Mongo's Burgers"}) +You can also pass an instance of a custom class to the ``insert_one()`` method. +This provides additional type safety if you're using a type-checking +tool. The instance you pass must inherit from the ``TypedDict`` class. + +.. include:: /includes/type-hints/typeddict-availability.rst + +The following example passes an instance of the ``Restaurant`` class to the ``insert_one()`` +method for added type safety: + +.. code-block:: python + :copyable: true + + class Restaurant(TypedDict): + name: str + + sample_restaurants.restaurants.insert_one(Movie(name="Mongo's Burgers") + Insert Multiple Documents ------------------------- @@ -78,6 +95,28 @@ The following example inserts a list of documents into the ``restaurants`` colle sample_restaurants.restaurants.insert_many(document_list) +You can also pass a list of instances of a custom class to the ``insert_many()`` method. +This provides additional type safety if you're using a type-checking +tool. The instances you pass must inherit from the ``TypedDict`` class. + +.. include:: /includes/type-hints/typeddict-availability.rst + +The following example calls the ``insert_many()`` method and passes a list that contains +instances of the ``Restaurant`` class. This adds type safety to the insert operation. + +.. code-block:: python + :copyable: true + + class Restaurant(TypedDict): + name: str + + document_list = [ + Restaurant(name="Mongo's Burgers"), + Restaurant(name="Mongo's Pizza") + ] + + sample_restaurants.restaurants.insert_many(document_list) + Modify Insert Behavior ---------------------- @@ -140,41 +179,6 @@ document-level validation. sample_restaurants.restaurants.insert_many(document_list, bypass_document_validation = True) -Type Hints ----------- - -.. include:: /includes/type-hints/intro.rst - -When you call the ``insert_one()`` or ``insert_many()`` method, you can pass one or more -instances of a custom class that represents the documents in the collection. - -To create -a custom type, define a class that inherits from the ``TypedDict`` class. In the class, -add a property for each field in the document. - -After you define the class, you can insert instances of the class. The following example -defines a ``Movie`` class to represent documents from the -``sample_mflix.movies`` collection. -Each ``Movie`` object contains two key-value pairs: ``name``, a string key with a string -value, and ``year``, a string key with an integer value. -The code then uses the ``insert_one()`` method to insert a ``Movie`` object. - -.. code-block:: python - :emphasize-lines: 10 - - from typing import TypedDict - - class Movie(TypedDict): - name: str - year: int - - client: MongoClient = MongoClient() - database = client["test_database"] - collection: Collection[Movie] = database["test_collection"] - inserted = collection.insert_one(Movie(name="Jurassic Park", year=1993)) - -.. include:: /includes/type-hints/tip-more-info.rst - Troubleshooting --------------- @@ -308,6 +312,7 @@ The following code example shows how to implement this solution: The ``NotRequired`` class is available only in Python 3.11 and later. To use ``NotRequired`` in earlier versions of Python, install the ``typing_extensions`` package instead. + Additional Information ---------------------- From 4a208eebb4599d723503a6ec260995a3a580de3d Mon Sep 17 00:00:00 2001 From: Mike Woofter <108414937+mongoKart@users.noreply.github.com> Date: Thu, 6 Feb 2025 14:08:02 -0600 Subject: [PATCH 04/10] fixes --- source/includes/type-hints/typeddict-availability.rst | 4 ++-- source/run-command.txt | 4 +++- source/write/bulk-write.txt | 8 ++++++-- source/write/insert.txt | 8 ++++++-- 4 files changed, 17 insertions(+), 7 deletions(-) diff --git a/source/includes/type-hints/typeddict-availability.rst b/source/includes/type-hints/typeddict-availability.rst index 98c34411..ac508465 100644 --- a/source/includes/type-hints/typeddict-availability.rst +++ b/source/includes/type-hints/typeddict-availability.rst @@ -1,3 +1,3 @@ -The examples in this section use the ``TypedDict`` class from the ``typing`` module. This -module is available only in Python 3.8 and later. To use the ``TypedDict`` class in +The ``TypedDict`` class is in the ``typing`` module, which +is available only in Python 3.8 and later. To use the ``TypedDict`` class in earlier versions of Python, install the ``typing_extensions`` package. \ No newline at end of file diff --git a/source/run-command.txt b/source/run-command.txt index 59ae9e38..89fcfee1 100644 --- a/source/run-command.txt +++ b/source/run-command.txt @@ -238,7 +238,9 @@ the class name. The class can be one of the following types: parameter, you must also include the class in a type hint for your ``CodecOptions`` object. -.. include:: /includes/type-hints/typeddict-availability.rst +.. note:: TypedDict in Python 3.7 and Earlier + + .. include:: /includes/type-hints/typeddict-availability.rst The following example decodes the BSON returned by the ``ping`` command to instances of the ``RawBSONDocument`` class: diff --git a/source/write/bulk-write.txt b/source/write/bulk-write.txt index 61cd234f..933cb97c 100644 --- a/source/write/bulk-write.txt +++ b/source/write/bulk-write.txt @@ -99,7 +99,9 @@ You can also create an instance of ``InsertOne`` by passing an instance of a cus to the constructor. This provides additional type safety if you're using a type-checking tool. The instance you pass must inherit from the ``TypedDict`` class. -.. include:: /includes/type-hints/typeddict-availability.rst +.. note:: TypedDict in Python 3.7 and Earlier + + .. include:: /includes/type-hints/typeddict-availability.rst The following example constructs an ``InsertOne`` instance by using a custom class for added type safety: @@ -176,7 +178,9 @@ You can also create an instance of ``ReplaceOne`` by passing an instance of a cu to the constructor. This provides additional type safety if you're using a type-checking tool. The instance you pass must inherit from the ``TypedDict`` class. -.. include:: /includes/type-hints/typeddict-availability.rst +.. note:: TypedDict in Python 3.7 and Earlier + + .. include:: /includes/type-hints/typeddict-availability.rst The following example constructs a ``ReplaceOne`` instance by using a custom class for added type safety: diff --git a/source/write/insert.txt b/source/write/insert.txt index 2790b244..67a00762 100644 --- a/source/write/insert.txt +++ b/source/write/insert.txt @@ -64,7 +64,9 @@ You can also pass an instance of a custom class to the ``insert_one()`` method. This provides additional type safety if you're using a type-checking tool. The instance you pass must inherit from the ``TypedDict`` class. -.. include:: /includes/type-hints/typeddict-availability.rst +.. note:: TypedDict in Python 3.7 and Earlier + + .. include:: /includes/type-hints/typeddict-availability.rst The following example passes an instance of the ``Restaurant`` class to the ``insert_one()`` method for added type safety: @@ -99,7 +101,9 @@ You can also pass a list of instances of a custom class to the ``insert_many()`` This provides additional type safety if you're using a type-checking tool. The instances you pass must inherit from the ``TypedDict`` class. -.. include:: /includes/type-hints/typeddict-availability.rst +.. note:: TypedDict in Python 3.7 and Earlier + + .. include:: /includes/type-hints/typeddict-availability.rst The following example calls the ``insert_many()`` method and passes a list that contains instances of the ``Restaurant`` class. This adds type safety to the insert operation. From c420d12aa4e6c5ba4cf4986b960779d19e94b8b0 Mon Sep 17 00:00:00 2001 From: Mike Woofter <108414937+mongoKart@users.noreply.github.com> Date: Tue, 11 Feb 2025 17:22:27 -0600 Subject: [PATCH 05/10] rm feedback --- source/connect/mongoclient.txt | 8 +++++++- source/includes/type-hints/intro.rst | 2 +- source/includes/type-hints/tip-more-info.rst | 7 ------- .../includes/type-hints/tip-type-checkers.rst | 4 ++++ .../troubleshooting-incompatible-type.rst | 5 +++-- .../type-hints/typeddict-availability.rst | 6 ++++-- source/read/retrieve.txt | 18 ------------------ source/write/bulk-write.txt | 2 ++ source/write/insert.txt | 14 ++++++++++---- 9 files changed, 31 insertions(+), 35 deletions(-) delete mode 100644 source/includes/type-hints/tip-more-info.rst create mode 100644 source/includes/type-hints/tip-type-checkers.rst diff --git a/source/connect/mongoclient.txt b/source/connect/mongoclient.txt index aa5d9498..21ccf03f 100644 --- a/source/connect/mongoclient.txt +++ b/source/connect/mongoclient.txt @@ -137,7 +137,12 @@ constructor accepts. All parameters are optional. - ``bson.raw_bson.RawBSONDocument``. - A subclass of the ``collections.abc.Mapping`` type, such as ``bson.son.SON``. If you specify ``bson.son.SON`` as the document class, you must also specify types - for the key and value. + for the key and value, as shown in the following example: + + .. code-block:: python + + client = MongoClient(document_class=SON[str, int]) + - A subclass of the ``TypedDict`` type. To pass a ``TypedDict`` subclass for this parameter, you must also include the class in a type hint for your ``MongoClient`` object, as shown in the following example: @@ -149,6 +154,7 @@ constructor accepts. All parameters are optional. .. include:: /includes/type-hints/typeddict-availability.rst **Data type:** ``Type[_DocumentType]`` + **Default:** ``dict`` * - ``tz_aware`` - If this parameter is ``True``, the client treats ``datetime`` values as aware. diff --git a/source/includes/type-hints/intro.rst b/source/includes/type-hints/intro.rst index 605f3118..4aa6a0b6 100644 --- a/source/includes/type-hints/intro.rst +++ b/source/includes/type-hints/intro.rst @@ -1,6 +1,6 @@ If your application uses Python 3.5 or later, you can add *type hints*, as described in `PEP 484 `__, to your code. Type hints denote the data types of variables, parameters, and function return -values, as well as the structure of documents. +values, and the structure of documents. Some IDEs can use type hints to check your code for type errors and suggest appropriate options for code completion. \ No newline at end of file diff --git a/source/includes/type-hints/tip-more-info.rst b/source/includes/type-hints/tip-more-info.rst deleted file mode 100644 index 64482e8e..00000000 --- a/source/includes/type-hints/tip-more-info.rst +++ /dev/null @@ -1,7 +0,0 @@ -.. tip:: Learn More About Type Hints - - See the following resources for more information about type hints: - - - `typing `__ module (Python 3.5+) - - `mypy `__ - - `typing_extensions `__ package \ No newline at end of file diff --git a/source/includes/type-hints/tip-type-checkers.rst b/source/includes/type-hints/tip-type-checkers.rst new file mode 100644 index 00000000..46f73571 --- /dev/null +++ b/source/includes/type-hints/tip-type-checkers.rst @@ -0,0 +1,4 @@ +.. tip:: Type-Checking Tools + + To learn more about type-checking tools available for Python, see + :ref:`Type Checkers ` on the Tools page. \ No newline at end of file diff --git a/source/includes/type-hints/troubleshooting-incompatible-type.rst b/source/includes/type-hints/troubleshooting-incompatible-type.rst index 4abb01bc..dc7e8fbd 100644 --- a/source/includes/type-hints/troubleshooting-incompatible-type.rst +++ b/source/includes/type-hints/troubleshooting-incompatible-type.rst @@ -1,8 +1,9 @@ Incompatible Type ~~~~~~~~~~~~~~~~~ -If you use the generic form of the ``MongoClient`` class, your type checker might -show an error similar to the following: +If you specify ``MongoClient`` as a type hint but don't include data types for +the document, keys, and values, your type checker might show an error similar to +the following: .. code-block:: python diff --git a/source/includes/type-hints/typeddict-availability.rst b/source/includes/type-hints/typeddict-availability.rst index ac508465..2898e590 100644 --- a/source/includes/type-hints/typeddict-availability.rst +++ b/source/includes/type-hints/typeddict-availability.rst @@ -1,3 +1,5 @@ -The ``TypedDict`` class is in the ``typing`` module, which +The `TypedDict `__ class +is in the ``typing`` module, which is available only in Python 3.8 and later. To use the ``TypedDict`` class in -earlier versions of Python, install the ``typing_extensions`` package. \ No newline at end of file +earlier versions of Python, install the +`typing_extensions `__ package. \ No newline at end of file diff --git a/source/read/retrieve.txt b/source/read/retrieve.txt index 907995e3..9aa955a1 100644 --- a/source/read/retrieve.txt +++ b/source/read/retrieve.txt @@ -199,24 +199,6 @@ to the ``find()`` method: .. include:: /includes/collation-override-note.rst -Troubleshooting ---------------- - -Modifying Raw BSON -~~~~~~~~~~~~~~~~~~ - -Instances of the ``RawBSONDocument`` class are read-only. If you try to set a value on a -``RawBSONDocument`` object, you will see an error similar to one of the following: - -.. code-block:: python - - error: Unsupported target for indexed assignment - ("RawBSONDocument") [index] - -.. code-block:: python - - TypeError: 'RawBSONDocument' object does not support item assignment - .. _pymongo-retrieve-additional-information: Additional Information diff --git a/source/write/bulk-write.txt b/source/write/bulk-write.txt index 933cb97c..4e4579b4 100644 --- a/source/write/bulk-write.txt +++ b/source/write/bulk-write.txt @@ -193,6 +193,8 @@ class for added type safety: To replace multiple documents, you must create an instance of ``ReplaceOne`` for each document. +.. include:: /includes/type-hints/tip-type-checkers.rst + Delete Operations ~~~~~~~~~~~~~~~~~ diff --git a/source/write/insert.txt b/source/write/insert.txt index 67a00762..39b41cc8 100644 --- a/source/write/insert.txt +++ b/source/write/insert.txt @@ -77,7 +77,9 @@ method for added type safety: class Restaurant(TypedDict): name: str - sample_restaurants.restaurants.insert_one(Movie(name="Mongo's Burgers") + sample_restaurants.restaurants.insert_one(Restaurant(name="Mongo's Burgers") + +.. include:: /includes/type-hints/tip-type-checkers.rst Insert Multiple Documents ------------------------- @@ -121,6 +123,8 @@ instances of the ``Restaurant`` class. This adds type safety to the insert opera sample_restaurants.restaurants.insert_many(document_list) +.. include:: /includes/type-hints/tip-type-checkers.rst + Modify Insert Behavior ---------------------- @@ -205,7 +209,7 @@ calling the ``insert_many()`` method instead. TypedDict Missing _id Key ~~~~~~~~~~~~~~~~~~~~~~~~~ -As discussed above, if you don't specify the ``_id`` field, {+driver-short+} automatically +If you don't specify the ``_id`` field, {+driver-short+} automatically inserts it into the document. You can retrieve the value of the ``_id`` field at runtime, but if you use MyPy or another tool to perform static type-checking, it won't find the ``_id`` field in your class and @@ -313,8 +317,10 @@ The following code example shows how to implement this solution: .. important:: NotRequired Requires Python 3.11+ - The ``NotRequired`` class is available only in Python 3.11 and later. - To use ``NotRequired`` in earlier versions of Python, install the ``typing_extensions`` + The `NotRequired `__ + class is available only in Python 3.11 and later. + To use ``NotRequired`` in earlier versions of Python, install the + `typing_extensions `__ package instead. Additional Information From 301d38a06fffbebf4a80f59dadd7e0eb120ad4e1 Mon Sep 17 00:00:00 2001 From: Mike Woofter <108414937+mongoKart@users.noreply.github.com> Date: Wed, 5 Mar 2025 14:54:17 -0600 Subject: [PATCH 06/10] tech review --- source/connect/mongoclient.txt | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/source/connect/mongoclient.txt b/source/connect/mongoclient.txt index 21ccf03f..602b6bd0 100644 --- a/source/connect/mongoclient.txt +++ b/source/connect/mongoclient.txt @@ -132,11 +132,13 @@ constructor accepts. All parameters are optional. * - ``document_class`` - The default class that the client uses to decode BSON documents returned by queries. - This parameter supports the following types: + This parameter accepts the following types: - - ``bson.raw_bson.RawBSONDocument``. + - ``bson.raw_bson.RawBSONDocument``. To learn more about the ``RawBSONDocument`` class, + see :ref:`pymongo-bson-raw`. + - A subclass of the ``collections.abc.Mapping`` type, such as ``bson.son.SON``. - If you specify ``bson.son.SON`` as the document class, you must also specify types + If you specify ``bson.son.SON`` as the document class, you may also specify types for the key and value, as shown in the following example: .. code-block:: python @@ -265,6 +267,17 @@ The following example shows how to include this data type in your type annotatio from typing import Any, Dict client: MongoClient[Dict[str, Any]] = MongoClient() +If all the documents that you are working with correspond to a single custom type, you +can specify the custom type as a type hint for your ``MongoClient`` object. This +provides more accurate type information than the generic ``Dict[str, Any]`` type. + +The following example shows how to specify the ``Movie`` type as a type hint for a +``MongoClient`` object: + +.. code-block:: python + + client: MongoClient[Movie] = MongoClient() + Troubleshooting --------------- From c6bd034f7dcba95e9f8fbbaab718e201ed52f9cd Mon Sep 17 00:00:00 2001 From: Mike Woofter <108414937+mongoKart@users.noreply.github.com> Date: Wed, 5 Mar 2025 15:15:08 -0600 Subject: [PATCH 07/10] add link to rawbson docs --- source/run-command.txt | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/source/run-command.txt b/source/run-command.txt index 89fcfee1..465a8eb5 100644 --- a/source/run-command.txt +++ b/source/run-command.txt @@ -230,7 +230,8 @@ The ``Database.command()`` method can decode the returned BSON documents to inst of a specific class. To specify this class, construct a ``CodecOptions`` object and pass the class name. The class can be one of the following types: -- ``bson.raw_bson.RawBSONDocument``. +- ``bson.raw_bson.RawBSONDocument``. To learn more about the ``RawBSONDocument`` class, + see :ref:`pymongo-bson-raw`. - A subclass of the ``collections.abc.Mapping`` type, such as ``bson.son.SON``. If you specify ``bson.son.SON`` as the document class, you must also specify types for the key and value. From 7f3d5609ad37111c945416afaece931a00528d10 Mon Sep 17 00:00:00 2001 From: Mike Woofter <108414937+mongoKart@users.noreply.github.com> Date: Wed, 5 Mar 2025 15:38:13 -0600 Subject: [PATCH 08/10] add link ref --- source/data-formats/bson.txt | 2 ++ 1 file changed, 2 insertions(+) diff --git a/source/data-formats/bson.txt b/source/data-formats/bson.txt index 3a91f882..79d39037 100644 --- a/source/data-formats/bson.txt +++ b/source/data-formats/bson.txt @@ -117,6 +117,8 @@ The following example reads the sample BSON document from ``file.bson``: {"address": {"street": "Pizza St", "zipcode": "10003"}, "coord": [-73.982419, 41.579505], "cuisine": "Pizza", "name": "Mongo's Pizza"} +.. _pymongo-bson-raw: + Work with Raw BSON Data ----------------------- From 9a3333194ae53309e8d2c9c595266fb209e6e12e Mon Sep 17 00:00:00 2001 From: Mike Woofter <108414937+mongoKart@users.noreply.github.com> Date: Thu, 6 Mar 2025 16:48:39 -0600 Subject: [PATCH 09/10] feedback --- .../aggregation/aggregation-tutorials/filtered-subset.txt | 4 +--- source/connect/mongoclient.txt | 8 ++++---- source/run-command.txt | 6 +++--- 3 files changed, 8 insertions(+), 10 deletions(-) diff --git a/source/aggregation/aggregation-tutorials/filtered-subset.txt b/source/aggregation/aggregation-tutorials/filtered-subset.txt index 8bf3c50c..e9a738b1 100644 --- a/source/aggregation/aggregation-tutorials/filtered-subset.txt +++ b/source/aggregation/aggregation-tutorials/filtered-subset.txt @@ -90,9 +90,7 @@ Tutorial Next, add a :manual:`$sort ` stage that sorts the documents in descending order by the ``dateofbirth`` field to - list the youngest people first. Because Python dictionaries don't maintain the - order of their elements, use a ``SON``or ``OrderedDict`` object - instead: + list the youngest people first: .. literalinclude:: /includes/aggregation/filtered-subset.py :language: python diff --git a/source/connect/mongoclient.txt b/source/connect/mongoclient.txt index 602b6bd0..c7209bd9 100644 --- a/source/connect/mongoclient.txt +++ b/source/connect/mongoclient.txt @@ -137,13 +137,13 @@ constructor accepts. All parameters are optional. - ``bson.raw_bson.RawBSONDocument``. To learn more about the ``RawBSONDocument`` class, see :ref:`pymongo-bson-raw`. - - A subclass of the ``collections.abc.Mapping`` type, such as ``bson.son.SON``. - If you specify ``bson.son.SON`` as the document class, you may also specify types - for the key and value, as shown in the following example: + - A subclass of the ``collections.abc.Mapping`` type, such as ``OrderedDict``. + Depending on the strictness of your type-checking rules, you might also need to + specify types for the key and value, as shown in the following example: .. code-block:: python - client = MongoClient(document_class=SON[str, int]) + client = MongoClient(document_class=OrderedDict[str, int]) - A subclass of the ``TypedDict`` type. To pass a ``TypedDict`` subclass for this parameter, you must also include the class in a type hint for your ``MongoClient`` diff --git a/source/run-command.txt b/source/run-command.txt index 465a8eb5..22296294 100644 --- a/source/run-command.txt +++ b/source/run-command.txt @@ -232,9 +232,9 @@ the class name. The class can be one of the following types: - ``bson.raw_bson.RawBSONDocument``. To learn more about the ``RawBSONDocument`` class, see :ref:`pymongo-bson-raw`. -- A subclass of the ``collections.abc.Mapping`` type, such as ``bson.son.SON``. - If you specify ``bson.son.SON`` as the document class, you must also specify types - for the key and value. +- A subclass of the ``collections.abc.Mapping`` type, such as ``OrderedDict``. + Depending on the strictness of your type-checking rules, you might also need to + specify types for the key and value. - A subclass of the ``TypedDict`` type. To pass a ``TypedDict`` subclass for this parameter, you must also include the class in a type hint for your ``CodecOptions`` object. From 013530d1e5e5dd0acf0d262d837e65b6e28edd9a Mon Sep 17 00:00:00 2001 From: Mike Woofter <108414937+mongoKart@users.noreply.github.com> Date: Mon, 10 Mar 2025 13:06:26 -0500 Subject: [PATCH 10/10] add movie definition --- source/connect/mongoclient.txt | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/source/connect/mongoclient.txt b/source/connect/mongoclient.txt index c7209bd9..c2b31266 100644 --- a/source/connect/mongoclient.txt +++ b/source/connect/mongoclient.txt @@ -276,6 +276,12 @@ The following example shows how to specify the ``Movie`` type as a type hint for .. code-block:: python + from typing import TypedDict + + class Movie(TypedDict): + name: str + year: int + client: MongoClient[Movie] = MongoClient() Troubleshooting