Merge branch 'factors' of https://github.com/pyc-team/pytorch_concepts into factors

Author: GiuseppeMarra

README.md

@@ -1,5 +1,5 @@
 <p align="center">
-  <img src="doc/_static/img/pyc_logo.png" alt="PyC Logo" width="40%">
+  <img src="https://raw.githubusercontent.com/pyc-team/pytorch_concepts/refs/heads/factors/doc/_static/img/pyc_logo.png" alt="PyC Logo" width="40%">
 </p>
 
 <p align="center">
@@ -10,12 +10,12 @@
 </p>
 
 <p align="center">
-  <a href="https://pytorch-concepts.readthedocs.io/en/latest/guides/installation.html">🚀 Getting Started</a> - 
-  <a href="https://pytorch-concepts.readthedocs.io/">📚 Documentation</a> - 
+  <a href="https://pytorch-concepts.readthedocs.io/en/latest/guides/installation.html">🚀 Getting Started</a> -
+  <a href="https://pytorch-concepts.readthedocs.io/">📚 Documentation</a> -
   <a href="https://pytorch-concepts.readthedocs.io/en/latest/guides/using.html">💻 User guide</a>
 </p>
 
-<img src="doc/_static/img/logos/pyc.svg" width="20px" align="center"> PyC is a library built upon <img src="doc/_static/img/logos/pytorch.svg" width="20px" align="center"> PyTorch and <img src="doc/_static/img/logos/lightning.svg" width="20px" align="center"> Pytorch Lightning to easily implement **interpretable and causally transparent deep learning models**.
+<img src="https://raw.githubusercontent.com/pyc-team/pytorch_concepts/refs/heads/factors/doc/_static/img/logos/pyc.svg" width="20px"> PyC is a library built upon <img src="https://raw.githubusercontent.com/pyc-team/pytorch_concepts/refs/heads/factors/doc/_static/img/logos/pytorch.svg" width="20px" align="center"> PyTorch and <img src="https://raw.githubusercontent.com/pyc-team/pytorch_concepts/refs/heads/factors/doc/_static/img/logos/lightning.svg" width="20px" align="center"> Pytorch Lightning to easily implement **interpretable and causally transparent deep learning models**.
 The library provides primitives for layers (encoders, predictors, special layers), probabilistic models, and APIs for running experiments at scale.
 
 The name of the library stands for both
@@ -26,7 +26,7 @@ The name of the library stands for both
 
 # Quick Start
 
-You can install PyC with core dependencies from [PyPI](https://pypi.org/project/pytorch-concepts/):
+You can install <img src="https://raw.githubusercontent.com/pyc-team/pytorch_concepts/refs/heads/factors/doc/_static/img/logos/pyc.svg" width="20px"> PyC with core dependencies from [PyPI](https://pypi.org/project/pytorch-concepts/):
 
 ```bash
 pip install pytorch-concepts
@@ -38,19 +38,19 @@ After installation, you can import it in your Python scripts as:
 import torch_concepts as pyc
 ```
 
-Follow our [user guide](https://pytorch-concepts.readthedocs.io/en/latest/guides/using.html) to get started with building interpretable models using PyC!
+Follow our [user guide](https://pytorch-concepts.readthedocs.io/en/latest/guides/using.html) to get started with building interpretable models using <img src="https://raw.githubusercontent.com/pyc-team/pytorch_concepts/refs/heads/factors/doc/_static/img/logos/pyc.svg" width="20px"> PyC!
 
 ---
 
-# <img src="doc/_static/img/logos/pyc.svg" width="20px" align="center"> PyC Software Stack
+# <img src="https://raw.githubusercontent.com/pyc-team/pytorch_concepts/refs/heads/factors/doc/_static/img/logos/pyc.svg" width="20px"> PyC Software Stack
 The library is organized to be modular and accessible at different levels of abstraction:
-- <img src="doc/_static/img/logos/conceptarium.svg" width="20px" align="center"> **Conceptarium (No-code API). Use case: applications and benchmarking.** These APIs allow to easily run large-scale highly parallelized and standardized experiments by interfacing with configuration files. Built on top of <img src="doc/_static/img/logos/hydra-head.svg" width="20px" align="center"> Hydra and <img src="doc/_static/img/logos/wandb.svg" width="20px" align="center"> WandB.
-- **High-level APIs. Use case: use out-of-the-box state-of-the-art models.** These APIs allow to instantiate use implemented models with 1 line of code. This interface is built in <img src="doc/_static/img/logos/lightning.svg" width="20px" align="center"> Pytorch Lightning to easily standardize training and evaluation.
+- <img src="https://raw.githubusercontent.com/pyc-team/pytorch_concepts/refs/heads/factors/doc/_static/img/logos/conceptarium.svg" width="20px" align="center"> **Conceptarium (No-code API). Use case: applications and benchmarking.** These APIs allow to easily run large-scale highly parallelized and standardized experiments by interfacing with configuration files. Built on top of <img src="https://raw.githubusercontent.com/pyc-team/pytorch_concepts/refs/heads/factors/doc/_static/img/logos/hydra-head.svg" width="20px" align="center"> Hydra and <img src="https://raw.githubusercontent.com/pyc-team/pytorch_concepts/refs/heads/factors/doc/_static/img/logos/wandb.svg" width="20px" align="center"> WandB.
+- **High-level APIs. Use case: use out-of-the-box state-of-the-art models.** These APIs allow to instantiate use implemented models with 1 line of code. This interface is built in <img src="https://raw.githubusercontent.com/pyc-team/pytorch_concepts/refs/heads/factors/doc/_static/img/logos/lightning.svg" width="20px" align="center"> Pytorch Lightning to easily standardize training and evaluation.
 - **Mid-level APIs. Use case: build custom interpretable and causally transparent probabilistic graphical models.** These APIs allow to build new interpretable probabilistic models and run efficient tensorial probabilistic inference.
-- **Low-level APIs. Use case: assemble custom interpretable architectures.** These APIs allow to build architectures from basic interpretable layers in a plain <img src="doc/_static/img/logos/pytorch.svg" width="20px" align="center"> PyTorch-like interface. These APIs also include metrics, losses, and datasets.
+- **Low-level APIs. Use case: assemble custom interpretable architectures.** These APIs allow to build architectures from basic interpretable layers in a plain <img src="https://raw.githubusercontent.com/pyc-team/pytorch_concepts/refs/heads/factors/doc/_static/img/logos/pytorch.svg" width="20px" align="center"> PyTorch-like interface. These APIs also include metrics, losses, and datasets.
 
 <p align="center">
-  <img src="doc/_static/img/pyc_software_stack.png" alt="PyC Software Stack" width="90%">
+  <img src="https://raw.githubusercontent.com/pyc-team/pytorch_concepts/refs/heads/factors/doc/_static/img/pyc_software_stack.png" alt="PyC Software Stack" width="90%">
 </p>
 
 ---
@@ -96,9 +96,10 @@ Reference authors: [Pietro Barbiero](http://www.pietrobarbiero.eu/), [Giovanni D
 This project is supported by the following organizations:
 
 <p align="center">
-  <img src="doc/_static/img/funding/fwo_kleur.png" alt="FWO - Research Foundation Flanders" height="60" style="margin: 20px;">
+  <img src="https://raw.githubusercontent.com/pyc-team/pytorch_concepts/refs/heads/factors/doc/_static/img/funding/fwo_kleur.png" alt="FWO - Research Foundation Flanders" height="60" style="margin: 20px;">
   &nbsp;&nbsp;&nbsp;&nbsp;
-  <img src="doc/_static/img/funding/hasler.png" alt="Hasler Foundation" height="60" style="margin: 20px;">
+  <img src="https://raw.githubusercontent.com/pyc-team/pytorch_concepts/refs/heads/factors/doc/_static/img/funding/hasler.png" alt="Hasler Foundation" height="60" style="margin: 20px;">
   &nbsp;&nbsp;&nbsp;&nbsp;
-  <img src="doc/_static/img/funding/snsf.png" alt="SNSF - Swiss National Science Foundation" height="60" style="margin: 20px;">
+  <img src="https://raw.githubusercontent.com/pyc-team/pytorch_concepts/refs/heads/factors/doc/_static/img/funding/snsf.png" alt="SNSF - Swiss National Science Foundation" height="60" style="margin: 20px;">
 </p>
+

doc/guides/using_low_level.rst

@@ -66,16 +66,23 @@ takes as input both ``Endogenous`` and ``Exogenous`` representations and produce
 
 .. code-block:: python
 
- pyc.nn.HyperLinearCUC(in_features_endogenous=10, in_features_exogenous=7,
-                       embedding_size=24, out_features=3)
+ pyc.nn.HyperLinearCUC(
+    in_features_endogenous=10,
+    in_features_exogenous=7,
+    embedding_size=24,
+    out_features=3
+ )
 
 As a final example, graph learners are a special layers that learn relationships between concepts.
 They do not follow the standard naming convention of encoders and predictors, but their purpose should be
 clear from their name.
 
 .. code-block:: python
 
- wanda = pyc.nn.WANDAGraphLearner(['c1', 'c2', 'c3'], ['task A', 'task B', 'task C'])
+ wanda = pyc.nn.WANDAGraphLearner(
+    ['c1', 'c2', 'c3'],
+    ['task A', 'task B', 'task C']
+ )
 
 
 Step 1: Import Libraries
@@ -152,9 +159,7 @@ Train with both concept and task supervision:
    import torch.nn.functional as F
 
    # Compute losses
-   concept_loss = F.binary_cross_entropy_with_endogenous(
-       concept_endogenous, concept_labels
-   )
+   concept_loss = F.binary_cross_entropy(torch.sigmoid(concept_endogenous), concept_labels)
    task_loss = F.cross_entropy(task_endogenous, task_labels)
    total_loss = task_loss + 0.5 * concept_loss
 
@@ -183,9 +188,11 @@ The context manager takes two main arguments: **strategies** and **policies**.
    policy = UniformPolicy(out_features=n_concepts)
 
    # Apply intervention to encoder
-   with intervention(policies=policy,
-                     strategies=strategy,
-                     target_concepts=[0, 2]) as new_encoder_layer:
+   with intervention(
+       policies=policy,
+       strategies=strategy,
+       target_concepts=[0, 2]
+   ) as new_encoder_layer:
        intervened_concepts = new_encoder_layer(input=x)
        intervened_tasks = model['predictor'](endogenous=intervened_concepts)
 

doc/guides/using_mid_level_causal.rst

@@ -56,8 +56,8 @@ Structural Equation Models
   .. code-block:: python
 
      sem_model = ProbabilisticModel(
-         variables=[exogenous_var, genotype_var, ...],
-         parametric_cpds=[exogenous_cpd, genotype_cpd, ...]
+         variables=[exogenous_var, genotype_var],
+         parametric_cpds=[exogenous_cpd, genotype_cpd]
      )
 
 Interventions
@@ -78,9 +78,9 @@ For example, to set ``smoking`` to 0 (prevent smoking) and query the effect on d
    )
 
    with intervention(
-           policies=UniformPolicy(out_features=1),
-           strategies=smoking_strategy_0,
-           target_concepts=["smoking"]
+       policies=UniformPolicy(out_features=1),
+       strategies=smoking_strategy_0,
+       target_concepts=["smoking"]
    ):
        intervened_results_0 = inference_engine.query(
            query_concepts=["genotype", "smoking", "tar", "cancer"],
@@ -258,9 +258,9 @@ Perform do-interventions to estimate causal effects:
    )
 
    with intervention(
-           policies=UniformPolicy(out_features=1),
-           strategies=smoking_strategy_0,
-           target_concepts=["smoking"]
+       policies=UniformPolicy(out_features=1),
+       strategies=smoking_strategy_0,
+       target_concepts=["smoking"]
    ):
        intervened_results_0 = inference_engine.query(
            query_concepts=["genotype", "smoking", "tar", "cancer"],
@@ -275,9 +275,9 @@ Perform do-interventions to estimate causal effects:
    )
 
    with intervention(
-           policies=UniformPolicy(out_features=1),
-           strategies=smoking_strategy_1,
-           target_concepts=["smoking"]
+       policies=UniformPolicy(out_features=1),
+       strategies=smoking_strategy_1,
+       target_concepts=["smoking"]
    ):
        intervened_results_1 = inference_engine.query(
            query_concepts=["genotype", "smoking", "tar", "cancer"],

doc/guides/using_mid_level_proba.rst

@@ -31,22 +31,29 @@ At this API level, models are represented as probabilistic models where:
 
   .. code-block:: python
 
-     concepts = pyc.EndogenousVariable(concepts=["c1", "c2", "c3"], parents=[],
-                                       distribution=torch.distributions.RelaxedBernoulli)
+     concepts = pyc.EndogenousVariable(
+        concepts=["c1", "c2", "c3"],
+        parents=[],
+        distribution=torch.distributions.RelaxedBernoulli
+     )
 
 - ``ParametricCPD`` objects represent conditional probability distributions (CPDs) between variables in the probabilistic model and are parameterized by |pyc_logo| PyC layers. For instance we can define a list of three parametric CPDs for the above concepts as:
 
   .. code-block:: python
 
-     concept_cpd = pyc.nn.ParametricCPD(concepts=["c1", "c2", "c3"],
-                                        parametrization=pyc.nn.LinearZC(in_features=10, out_features=3))
+     concept_cpd = pyc.nn.ParametricCPD(
+        concepts=["c1", "c2", "c3"],
+        parametrization=pyc.nn.LinearZC(in_features=10, out_features=3)
+     )
 
 - ``ProbabilisticModel`` objects are a collection of variables and CPDs. For instance we can define a model as:
 
   .. code-block:: python
 
-     probabilistic_model = pyc.nn.ProbabilisticModel(variables=concepts,
-                                                     parametric_cpds=concept_cpd)
+     probabilistic_model = pyc.nn.ProbabilisticModel(
+        variables=concepts,
+        parametric_cpds=concept_cpd
+     )
 
 Inference
 ^^^^^^^^^
@@ -55,8 +62,11 @@ Inference is performed using efficient tensorial probabilistic inference algorit
 
 .. code-block:: python
 
-   inference_engine = pyc.nn.AncestralSamplingInference(probabilistic_model=probabilistic_model,
-                                                        graph_learner=wanda, temperature=1.)
+   inference_engine = pyc.nn.AncestralSamplingInference(
+       probabilistic_model=probabilistic_model,
+       graph_learner=wanda,
+       temperature=1.
+   )
    predictions = inference_engine.query(["c1"], evidence={'input': x})
 
 
@@ -203,9 +213,11 @@ Perform do-calculus interventions:
    )
 
    # Apply intervention to encoder
-   with intervention(policies=policy,
-                     strategies=strategy,
-                     target_concepts=["round", "smooth"]):
+   with intervention(
+       policies=policy,
+       strategies=strategy,
+       target_concepts=["round", "smooth"]
+   ):
        intervened_predictions = inference_engine.query(
            query_concepts=["round", "smooth", "bright", "class_A", "class_B"],
            evidence={'input': x}

doc/modules/low_level_api.rst

@@ -82,16 +82,24 @@ takes as input both ``Endogenous`` and ``Exogenous`` representations and produce
 
 .. code-block:: python
 
- pyc.nn.HyperLinearCUC(in_features_endogenous=10, in_features_exogenous=7,
-                       embedding_size=24, out_features=3)
+ pyc.nn.HyperLinearCUC(
+    in_features_endogenous=10,
+    in_features_exogenous=7,
+    embedding_size=24,
+    out_features=3
+ )
 
 As a final example, graph learners are a special layers that learn relationships between concepts.
 They do not follow the standard naming convention of encoders and predictors, but their purpose should be
 clear from their name.
 
 .. code-block:: python
 
- wanda = pyc.nn.WANDAGraphLearner(['c1', 'c2', 'c3'], ['task A', 'task B', 'task C'])
+ wanda = pyc.nn.WANDAGraphLearner(
+    ['c1', 'c2', 'c3'],
+    ['task A', 'task B', 'task C']
+ )
+
 
 Models
 ^^^^^^^^^^^
@@ -123,8 +131,10 @@ At this API level, there are two types of inference that can be performed:
 
   .. code-block:: python
 
-     int_strategy = pyc.nn.DoIntervention(model=concept_bottleneck_model["encoder"],
-                                          constants=-10)
+     int_strategy = pyc.nn.DoIntervention(
+        model=concept_bottleneck_model["encoder"],
+        constants=-10
+     )
 
   **Intervention Policies**: define the order/set of concepts to intervene on e.g., we can intervene on all concepts uniformly:
 
@@ -136,10 +146,13 @@ At this API level, there are two types of inference that can be performed:
 
   .. code-block:: python
 
-     with pyc.nn.intervention(policies=int_policy,
-                              strategies=int_strategy,
-                              target_concepts=[0, 2]) as new_encoder_layer:
-
+     with pyc.nn.intervention(
+        policies=int_policy,
+        strategies=int_strategy,
+        target_concepts=[0, 2]
+     ) as new_encoder_layer:
          endogenous_concepts = new_encoder_layer(input=x)
-         endogenous_tasks = concept_bottleneck_model['predictor'](endogenous=endogenous_concepts)
+         endogenous_tasks = concept_bottleneck_model['predictor'](
+            endogenous=endogenous_concepts
+         )
 

doc/modules/mid_level_api.rst

@@ -40,22 +40,29 @@ At this API level, models are represented as probabilistic models where:
 
   .. code-block:: python
 
-     concepts = pyc.EndogenousVariable(concepts=["c1", "c2", "c3"], parents=[],
-                                       distribution=torch.distributions.RelaxedBernoulli)
+     concepts = pyc.EndogenousVariable(
+        concepts=["c1", "c2", "c3"],
+        parents=[],
+        distribution=torch.distributions.RelaxedBernoulli
+     )
 
 - ``ParametricCPD`` objects represent conditional probability distributions (CPDs) between variables in the probabilistic model and are parameterized by |pyc_logo| PyC layers. For instance we can define a list of three parametric CPDs for the above concepts as:
 
   .. code-block:: python
 
-     concept_cpd = pyc.nn.ParametricCPD(concepts=["c1", "c2", "c3"],
-                                        parametrization=pyc.nn.LinearZC(in_features=10, out_features=3))
+     concept_cpd = pyc.nn.ParametricCPD(
+        concepts=["c1", "c2", "c3"],
+        parametrization=pyc.nn.LinearZC(in_features=10, out_features=3)
+     )
 
 - ``ProbabilisticModel`` objects are a collection of variables and CPDs. For instance we can define a model as:
 
   .. code-block:: python
 
-     probabilistic_model = pyc.nn.ProbabilisticModel(variables=concepts,
-                                                     parametric_cpds=concept_cpd)
+     probabilistic_model = pyc.nn.ProbabilisticModel(
+        variables=concepts,
+        parametric_cpds=concept_cpd
+     )
 
 Inference
 ^^^^^^^^^
@@ -64,8 +71,11 @@ Inference is performed using efficient tensorial probabilistic inference algorit
 
 .. code-block:: python
 
-   inference_engine = pyc.nn.AncestralSamplingInference(probabilistic_model=probabilistic_model,
-                                                        graph_learner=wanda, temperature=1.)
+   inference_engine = pyc.nn.AncestralSamplingInference(
+       probabilistic_model=probabilistic_model,
+       graph_learner=wanda,
+       temperature=1.
+   )
    predictions = inference_engine.query(["c1"], evidence={'input': x})
 
 
@@ -106,8 +116,8 @@ Structural Equation Models
   .. code-block:: python
 
      sem_model = ProbabilisticModel(
-         variables=[exogenous_var, genotype_var, ...],
-         parametric_cpds=[exogenous_cpd, genotype_cpd, ...]
+         variables=[exogenous_var, genotype_var],
+         parametric_cpds=[exogenous_cpd, genotype_cpd]
      )
 
 Interventions
@@ -128,9 +138,9 @@ For example, to set ``smoking`` to 0 (prevent smoking) and query the effect on d
    )
 
    with intervention(
-           policies=UniformPolicy(out_features=1),
-           strategies=smoking_strategy_0,
-           target_concepts=["smoking"]
+      policies=UniformPolicy(out_features=1),
+      strategies=smoking_strategy_0,
+      target_concepts=["smoking"]
    ):
        intervened_results_0 = inference_engine.query(
            query_concepts=["genotype", "smoking", "tar", "cancer"],