👌 IMPROVE: Support colon fenced directives

[nthiery]

This is a draft of fix for #13.

I don't really know what I am doing here. So please, anyone familiar with the mdformat / mdit code base improve! In particular, the list of supported directives is currently hardcoded which is ugly.
I have first tried to build on top of mdit_py_plugins.colon_fence instead of mdit_py_plugins.container. This would have avoided to handle metadata manually and removed the need to provide a fixed list of directives; however, the content of the directive is not parsed recursively with colon_fence.

Current limitation: yaml metadata delimited by --- is not supported.

[nthiery]

Branch updated. I don't know how to progress from there. Could we get feedback or review? Thanks in advance!

[KyleKing]

Hi @nthiery, I have contributor access and can help review and release changes, but I don't use MyST. From this comment #13 (comment) and reviewing the status of https://github.com/jupyter-book/mystmd/blob/f1ccba4513f5285d3e3b36a10228ddb4f5ec67ef/packages/myst-parser/README.md?plain=1#L100-L106, it seems like this should be included into the formatter

[KyleKing]

I don't have time now, but I think there might be an opportunity to submit changes to mdit-py-plugins to add a boolean use_regex or similar to avoid long lists like this

I think we could proceed with this list and replace it with a regex if that support is available later

On the PR more generally, there are a few blocking issues with formatting and testing that need to be resolved

[KyleKing]

Can you please run tox locally? This looks like this should be formatted by black and part of the issue in CI. Let me know if you run into any trouble with your local setup

[KyleKing]

I would want us to avoid the performance impact of creating each of these handlers

If we were to keep a hardcoded list, I would prefer to validate rather than search for specific matches (e.g. extract the directive name with regex like [\w\-:]+, then if present in the set of allowed names, proceed) - or use regex conditional to search all variations

But that's all blocked by the limitation of container_plugin being fixed strings only (no regex): https://github.com/executablebooks/mdit-py-plugins/blob/93fbf645108394bb10ecd392383e96c4e8902f30/mdit_py_plugins/container/index.py

[KyleKing]

Thanks for adding tests!

It looks like you may have some missing formatting, the repeating pattern is:

<title>
.
<pre>
.
<post>
.

[KyleKing]

So something like:

MyST colon fenced directive with title
.
:::{admonition} MyST colon fenced directive with a title
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor
incididunt ut labore et dolore magna aliqua.
:::
.
:::{admonition} MyST colon fenced directive with a title

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor
incididunt ut labore et dolore magna aliqua.

:::
.

MyST colon fenced directive with metadata
.
:::{admonition} MyST colon fenced directive with metadata
:class: foo
:truc: bla
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor
incididunt ut labore et dolore magna aliqua.
:::
.
:::{admonition} MyST colon fenced directive with metadata

:class: foo
:truc: bla
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor
incididunt ut labore et dolore magna aliqua.

:::
.

MyST colon fenced directive with nested directive
.
::::{admonition} Parent directive
:::{image} foo.png
:class: foo
:alt: An image
:::
::::
.
::::{admonition} Parent directive

:::{image} foo.png

:class: foo
:alt: An image

:::

::::
.

MyST colon fenced directive with multiple nested admonitions
.
::::{admonition} Multiple nested admonitions
:::{admonition}
First nested admonition content.
:::
:::{admonition}
Second nested admonition content.
:::
:::{admonition}
Third nested admonition content.
:::
::::
.
::::{admonition} Multiple nested admonitions

:::{admonition}

First nested admonition content.

:::

:::{admonition}

Second nested admonition content.

:::

:::{admonition}

Third nested admonition content.

:::

::::
.

MyST colon fenced directive with mixed content and nested directives
.
::::{hint} A hint with nested tips and paragraphs
This is some introductory text.
:::{tip}
A nested tip with content.
:::
More text between directives.
:::{tip}
Another nested tip.
:::
Concluding text.
::::
.
::::{hint} A hint with nested tips and paragraphs

This is some introductory text.

:::{tip}

A nested tip with content.

:::

More text between directives.

:::{tip}

Another nested tip.

:::

Concluding text.

::::
.

MyST colon fenced directive nested in list
.
- Item with directive
  :::{tip} Nested tip in list item
  Tip content inside a list item.
  :::
.
- Item with directive
  :::{tip} Nested tip in list item

  Tip content inside a list item.

  :::
.

[KyleKing]

I have something like this working locally:

_YAML_METADATA_PATTERN = re.compile(r"^:\w[\w-]*:\s+.*$", re.MULTILINE)

def container_renderer(
    node: RenderTreeNode, context: RenderContext, *args, **kwargs
) -> str:
    children = node.children or []
    parts = []

    if children and children[0].type == "paragraph":
        tokens = children[0].to_tokens()
        yaml_lines = []

        for token in tokens:
            if token.type in {"paragraph_open", "paragraph_close"}:
                continue
            if token.content and _YAML_METADATA_PATTERN.match(token.content.strip()):
                yaml_lines.append(token.content.strip())
            elif yaml_lines:
                break  # Stop at first non-YAML line

        if yaml_lines:
            parts.append("\n".join(yaml_lines))
            children = children[1:]

    if children:
        rendered_children = "\n\n".join(child.render(context) for child in children)
        parts.append(rendered_children)

    if parts:
        content = "\n\n".join(parts)
        return f"{node.markup}{node.info}\n\n{content}\n\n{node.markup}"
    return f"{node.markup}{node.info}\n{node.markup}"

...

for directive in CONTAINER_DIRECTIVES: # Just minor nits
    RENDERERS[f"container_{{{directive}}}"] = container_renderer