Fix derivesFrom false negative in provablyDisjointClasses

Before the addition of `SymDenotation#mayDeriveFrom`,
tests/neg/i17132.min.scala was unsoundly accepted without a type error
because `R[T]` is reduced to `Any` where `T <: P[Any]`
by testing `provablyDisjointClasses` before `P` is added as a baseClass of `Q`.

Now, a recursion overflows occurs because:
- reducing `R[T] := T match case Q[t] => R[t]; case ...` requires
- proving `T` disjoint from `Q[t]` where `T <: P[Any]`, which asks
- whether existsCommonBaseTypeWithDisjointArguments, which requires
- normalizing the type arg to the base class P for the class Q, i.e. R[t]
- ...

In short, despite the use of the pending set in provablyDisjoint, diverging is
still possible when there is "cycle" in the type arguments in the base classes,
(and where the pending set is thus reset for a separate normalization problem).

One could attempt to add some more logic to detect these loops
s.t. they are considered "stuck", as opposed to reporting an error.
But it isn't clear that there are any concrete motivations for this.

Author: EugeneFlesselle

compiler/src/dotty/tools/dotc/core/SymDenotations.scala

@@ -865,9 +865,20 @@ object SymDenotations {
      *  and is the denoting symbol also different from `Null` or `Nothing`?
      *  @note  erroneous classes are assumed to derive from all other classes
      *         and all classes derive from them.
+     *  @note may return a false negative when `this.info.isInstanceOf[TempClassInfo]`.
      */
     def derivesFrom(base: Symbol)(using Context): Boolean = false
 
+    /** Could `this` derive from `base` now or in the future.
+     *  For concistency with derivesFrom, The info is only forced when this is a ClassDenotation.
+     *  If the info is a TempClassInfo then the baseClassSet may be temporarily approximated as empty.
+     *  This is problematic when stability of `!derivesFrom(base)` is assumed for soundness,
+     *  e.g., in `TypeComparer#provablyDisjointClasses`.
+     *  @note may return a false positive when `this.info.isInstanceOf[TempClassInfo]`.
+     */
+    final def mayDeriveFrom(base: Symbol)(using Context): Boolean =
+      this.isInstanceOf[ClassDenotation] && (info.isInstanceOf[TempClassInfo] || derivesFrom(base))
+
     /** Is this a Scala or Java annotation ? */
     def isAnnotation(using Context): Boolean =
       isClass && (derivesFrom(defn.AnnotationClass) || is(JavaAnnotation))

compiler/src/dotty/tools/dotc/core/TypeComparer.scala

@@ -3230,7 +3230,7 @@ class TypeComparer(@constructorOnly initctx: Context) extends ConstraintHandling
       .filter(child => child.exists && child != cls)
 
     def eitherDerivesFromOther(cls1: Symbol, cls2: Symbol): Boolean =
-      cls1.derivesFrom(cls2) || cls2.derivesFrom(cls1)
+      cls1.mayDeriveFrom(cls2) || cls2.mayDeriveFrom(cls1)
 
     def smallestNonTraitBase(cls: Symbol): Symbol =
       val classes = if cls.isClass then cls.asClass.baseClasses else cls.info.classSymbols

tests/neg/i17132.min.check

@@ -0,0 +1,31 @@
+-- Error: tests/neg/i17132.min.scala:4:7 -------------------------------------------------------------------------------
+4 |class Q[T <: P[Any]] extends P[R[T]] // error
+  |       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+  |       Recursion limit exceeded.
+  |       Maybe there is an illegal cyclic reference?
+  |       If that's not the case, you could also try to increase the stacksize using the -Xss JVM option.
+  |       For the unprocessed stack trace, compile with -Xno-enrich-error-messages.
+  |       A recurring operation is (inner to outer):
+  |
+  |         reduce type  t match ...
+  |         reduce type  t match ...
+  |         reduce type  t match ...
+  |         reduce type  t match ...
+  |         reduce type  t match ...
+  |         reduce type  t match ...
+  |         reduce type  t match ...
+  |         reduce type  t match ...
+  |         reduce type  t match ...
+  |         reduce type  t match ...
+  |         ...
+  |
+  |         reduce type  t match ...
+  |         reduce type  t match ...
+  |         reduce type  t match ...
+  |         reduce type  t match ...
+  |         reduce type  t match ...
+  |         reduce type  t match ...
+  |         reduce type  t match ...
+  |         reduce type  t match ...
+  |         reduce type  t match ...
+  |         reduce type  T match ...

tests/neg/i17132.min.scala

@@ -0,0 +1,9 @@
+
+class P[T]
+//class Q[T] extends P[R[T]] // ok
+class Q[T <: P[Any]] extends P[R[T]] // error
+//type Q[T <: P[Any]] <: P[R[T]] // ok
+
+type R[U] = U match
+  case Q[t] => R[t]
+  case P[t] => t