20251102_00 Release v1.0.3

[unixwzrd]

20251102_00 Release 1.0.3

2025-11-02: filetree pattern normalization improvements

• Added normalized handling for regex-like include/exclude inputs in bin/filetree.py, covering .* tokens and trailing directory slashes regardless of whether the values come from CLI flags, environment variables, or config files.
• Factored shared pattern utilities (collect_pattern_tokens, normalize_patterns) so filters behave consistently across sources.
• Introduced directory allowlisting so that specifying include directories (e.g., src/) limits traversal to those trees instead of surfacing matches from every folder.
• Taught pattern normalization to interpret escaped regex tokens like .*\\.py, ensuring commands generated by tools such as genmd still match the intended files.
• Adjusted bin/genmd so the embedded filetree run changes into GENMD_BASE before traversal, fixing cases where the generated markdown only showed the root node.
• Added optional symlink traversal: when genmd is configured with follow_links, the generated filetree command now passes -L, and filetree’s tests cover symlinked directories.
• Replaced ad-hoc config handling in genmd with the shared load_config/dump_config utilities from bin/shinclude/config_lib.sh, keeping setup, manifest, and doc generation in sync.
• Improved bin/tokencount messaging so missing tiktoken/NLTK assets prompt the user with the exact installation steps instead of raising an uncaught import error.
• Added unit coverage in tests/test_filetree_patterns.py to lock in expected normalization behavior for .git, .py, and mixed include expressions.
• Updated docs/filetree.md with guidance and examples that show the new normalization rules in action.

---

This discussion was created from the release 20251102_00 Release v1.0.3.