INTPYTHON-743 Allow using MongoDB connection string in DATABASES["HOST"] #383

timgraham · 2025-08-29T22:55:47Z

No description provided.

aclark4life

LGTM, let's see if @ShaneHarvey has any feedback

ShaneHarvey

LGTM

aclark4life · 2025-09-04T21:10:33Z

.github/workflows/mongodb_settings.py

-        db_settings["OPTIONS"].update({"tls": True, "tlsAllowInvalidCertificates": True})
+    uri = parse_uri(mongodb_uri)
+    if uri.get("username") and uri.get("password"):
+        db_settings["OPTIONS"] = {"tls": True, "tlsAllowInvalidCertificates": True}


Can you add a comment here to explain what we are working around? It sounds like this could come out at some point based on @ShaneHarvey 's comment

I think one of the old concerns here was mixing of ssl and tls args. That should no longer be a problem if all drivers have dropped support for ssl=true.

Also maybe of note we're only using PyMongo's parse_uri to get the username and password if they are in the URI.

I thought the linked issue described the issue well, so I don't see a need to repeat it. As far as I understand, the workaround won't be necessary after that issue is closed.

It's still confusing to use PyMongo's parse_uri to get the username and password in the context of deprecating our parse_uri because we can't rely on PyMongo's parse_uri when the query string arguments aren't regular characters e.g. I think PyMongo is doing something like this:

>>> URI = "mongodb://example.com/search?time=10:30" >>> uri = parse_uri(URI) >>> uri.get("hour") 10 >>> uri.get("minute") 30

Given that we've spent a lot of time getting to the status quo with parse_uri I want to make sure we're not adding more confusion in merging this feature.

parse_uri is only used for MongoDB connection strings, it has nothing to do with http://.

This logic is copied from:

django-mongodb-backend/django_mongodb_backend/utils.py

Lines 59 to 60 in d7f8fd2

"USER": uri.get("username"),

"PASSWORD": uri.get("password"),

parse_uri is only used for MongoDB connection strings, it has nothing to do with http://.

I fixed the example but the scheme doesn't matter, whether it's mongodb://, https:// or alex:// the MongoDB URI spec was inspired by but does not conform to the URI spec. To make things even more confusing, we add in dj-database-url's stated purpose of "12 factor app support" and my stated purpose of django_mongodb_backend.parse_uri is like dj-database-url.

E.g.

Feature RFC 3986 MongoDB URI Difference 12‑Factor

One host in authority Yes Multiple hosts with commas Ensure your tooling/stack supports multi-host URIs when setting environment variables.

Reserved char encoding Must percent-encode Allowed unescaped in some drivers Always URL-encode usernames and passwords to avoid parsing errors in env var consumers.

Query param case sensitivity Case-sensitive Case-insensitive Use consistent lower-case keys in env vars for portability across drivers.

Semantics of query component Opaque to RFC MongoDB-specific options Document driver-specific options and keep them minimal in environment variables.

Scheme semantics Generic mongodb / mongodb+srv with DNS expansion Ensure DNS queries are allowed and resolvable in the target runtime environment.

</nit>

That said, we obviously only need to support the MongoDB URI and I think we are well on our way here with HOST.

It's still confusing to use PyMongo's parse_uri to get the username and password in the context of deprecating our parse_uri

Are you still confused? I'm confused about what you're confused about. This is the same parsing as before, but without the intermediate call to django_mongodb_backend.parse_uri().

Why are we calling PyMongo's parse_uri? Is there a scenario in which we don't have to do that and can that scenario happen now before we merge this.

To check if the URI contains a username and password (and therefore, this is an ssl connection). Jib authored this workaround. I can try to find another workaround or try to solve the issue in mongo-orchestration (I'm unsure if there is still a blocker there), but I don't see why this should hold up this PR.

django_mongodb_backend/base.py

docs/source/intro/configure.rst

aclark4life · 2025-09-04T21:29:36Z

docs/source/releases/5.2.x.rst

+------------
+
+- Allowed specifying the MongoDB connection string in ``DATABASES["HOST"]``,
+  eliminating the need to use :func:`~django_mongodb_backend.utils.parse_uri`


Until we are sure, I'd probably refrain from declaring that we've eliminated the need for parse_uri. For example, can this be described as implementing a similar feature set as dj-database-url and if so, is this approach something that could eliminate the need for that 3rd party library in Django (via some upstream changes to Django) ?

I think it's useful to inform users about the reason for the change so that they can try to migrate and let us know if there are any problems.

This is somewhat orthogonal to dj-database-url because it relies on the fact that MongoDB accepts a connection string as the host. Other database connectors like psycopg and mysqlclient don't necessarily work the same way.

There has been some discussion of trying to integrate of dj-database-url into Django. I don't think any work is imminent.

This is somewhat orthogonal to dj-database-url

Yes it is but parse_uri is not (orthogonal to dj-database-url). So this does not eliminate the need for parse_uri.

I'm not sure what you mean. Are you suggesting not to deprecate parse_uri? What use case for parse_uri do you have in mind?

I think it's useful to inform users about the reason for the change so that they can try to migrate and let us know if there are any problems.

It's useful to inform them of the change but as someone who knows the reason for the change I can't say it's a particularly compelling or easy reason to tell and I don't think we'll gain any clarity on that in the short term. I would much rather see something like:

Hey you can just add the URI to the HOST and still set NAME now! (parse_uri is deprecated).

Dont' declare parse_uri eliminated because it implements a slightly different use case which has not been addressed with this change.

So just to clarify once more: we cannot support the database name in the URI because we are not parsing the URI?

After more consideration, it seems feasible to support using the connection string's "defaultauthdb" as the database so that NAME isn't required. It's consistent with the fact that MongoClient.get_database() returns that db. The name does need to be parsed out because Django relies on settings_dict["NAME"] in django/db/backends/base/creation.py.

I think it should be good to go. I left this part as a separate commit to ease review.

I think it should be good to go. I left this part as a separate commit to ease review.

I'm not sure what you mean. If we don't support /dbname in the URI in this PR then this PR does not "eliminate the need for parse_uri" because parse_uri still supports that. If we add support for /dbname in the URI in this PR, then this PR does indeed eliminate the need for parse_uri, for all practical purposes as far as I can tell.

The middle ground is to merge this and not say anything about parse_uri except maybe that it's deprecated.

Support for /dbname is added in commit "parse db name out of connection string".

tests/backend_/test_base.py

aclark4life

LEBTM (even better), thanks!

timgraham force-pushed the connection-string-in-host branch from 59efa04 to 50d854b Compare August 29, 2025 23:57

timgraham changed the title ~~Use MongoDB connection string in DATABASES["HOST"]~~ Allow using MongoDB connection string in DATABASES["HOST"] Sep 2, 2025

timgraham force-pushed the connection-string-in-host branch 2 times, most recently from 8aca9ed to ab88385 Compare September 2, 2025 14:54

timgraham changed the title ~~Allow using MongoDB connection string in DATABASES["HOST"]~~ INTPYTHON-743 Allow using MongoDB connection string in DATABASES["HOST"] Sep 2, 2025

aclark4life mentioned this pull request Sep 2, 2025

INTPYTHON-732 Don't parse the query string args #381

Closed

timgraham force-pushed the connection-string-in-host branch 2 times, most recently from 963cd39 to 836603e Compare September 3, 2025 14:09

timgraham marked this pull request as ready for review September 3, 2025 14:10

timgraham requested a review from aclark4life September 3, 2025 14:47

aclark4life requested a review from ShaneHarvey September 3, 2025 15:57

aclark4life approved these changes Sep 3, 2025

View reviewed changes

ShaneHarvey approved these changes Sep 4, 2025

View reviewed changes

aclark4life requested changes Sep 4, 2025

View reviewed changes

tests/backend_/test_base.py Show resolved Hide resolved

timgraham force-pushed the connection-string-in-host branch 3 times, most recently from 2484ae4 to 155c639 Compare September 5, 2025 15:21

aclark4life self-requested a review September 5, 2025 16:26

aclark4life approved these changes Sep 5, 2025

View reviewed changes

timgraham added 2 commits September 8, 2025 05:44

Add tests for DatabaseWrapper.get_connection_params()

5982f6b

INTPYTHON-743 Allow using MongoDB connection string in DATABASES["HOST"]

862c2cd

timgraham force-pushed the connection-string-in-host branch from 155c639 to 862c2cd Compare September 8, 2025 09:45

timgraham merged commit 862c2cd into mongodb:main Sep 8, 2025
17 checks passed

timgraham deleted the connection-string-in-host branch September 8, 2025 10:18

	"USER": uri.get("username"),
	"PASSWORD": uri.get("password"),

Feature	RFC 3986	MongoDB URI Difference	12‑Factor
One host in authority	Yes	Multiple hosts with commas	Ensure your tooling/stack supports multi-host URIs when setting environment variables.
Reserved char encoding	Must percent-encode	Allowed unescaped in some drivers	Always URL-encode usernames and passwords to avoid parsing errors in env var consumers.
Query param case sensitivity	Case-sensitive	Case-insensitive	Use consistent lower-case keys in env vars for portability across drivers.
Semantics of query component	Opaque to RFC	MongoDB-specific options	Document driver-specific options and keep them minimal in environment variables.
Scheme semantics	Generic	`mongodb` / `mongodb+srv` with DNS expansion	Ensure DNS queries are allowed and resolvable in the target runtime environment.

INTPYTHON-743 Allow using MongoDB connection string in DATABASES["HOST"] #383

INTPYTHON-743 Allow using MongoDB connection string in DATABASES["HOST"] #383

Uh oh!

Conversation

timgraham commented Aug 29, 2025

Uh oh!

aclark4life left a comment

Choose a reason for hiding this comment

Uh oh!

ShaneHarvey left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aclark4life Sep 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

timgraham Sep 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

timgraham Sep 5, 2025 • edited by aclark4life Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

aclark4life left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

aclark4life Sep 4, 2025 •

edited

Loading

timgraham Sep 5, 2025 •

edited

Loading

timgraham Sep 5, 2025 •

edited by aclark4life

Loading