Deprecate x-extensible-enum

Implements #831.

Author: Paul Ebermann

chapters/compatibility.adoc

@@ -51,10 +51,10 @@ services in a backward-compatible way:
   make sure that all constraints are clearly defined in description.
 * `enum` ranges can be reduced when used as input parameters, only if the server
   is ready to accept and handle old range values too. The range can be reduced
-  when used as output parameters.
+  when used only as output parameters.
 * `enum` ranges cannot be extended when used for output parameters — clients may
-  not be prepared to handle it. However, enum ranges can be extended when used
-  for input parameters.
+  not be prepared to handle it. However, `enum` ranges can be extended when used
+  only for input parameters.
 * You <<112>> that are used for output parameters and likely to
   be extended with growing functionality. The API definition should be updated
   first before returning new values.
@@ -124,8 +124,11 @@ Service clients should apply the robustness principle:
   http://martinfowler.com/bliki/TolerantReader.html["TolerantReader"] post),
   i.e. ignore new fields but do not eliminate them from payload if needed for
   subsequent {PUT} requests.
-** Be prepared that {x-extensible-enum} return parameters (see <<112, rule 112>>) may deliver new values;
-  either be agnostic or provide default behavior for unknown values, and do not eliminate them.
+** Be prepared that "extensible enum" return parameters (see <<112, rule 112>>)
+  may deliver new values;
+  either be agnostic or provide default behavior for unknown values, and
+  do not eliminate them if needed for subsequent {PUT} requests.
+  (This means you can't simply map it to a limited enumeration type like a Java `enum`.)
 ** Be prepared to handle HTTP status codes not explicitly specified in endpoint
   definitions. Note also, that status codes are extensible. Default handling is
   how you would treat the corresponding {x00} code (see
@@ -289,46 +292,90 @@ level data structures, since they don't support compatible, future extensions.
 
 
 [#112]
-== {SHOULD} use open-ended list of values (`x-extensible-enum`) for enumeration types
+== {SHOULD} use open-ended list of values (via `examples`) for enumeration types
 
 JSON schema `enum` is per definition a closed set of values that is assumed to be
-complete and not intended for extension. This closed principle of enumerations
-imposes compatibility issues when an enumeration must be extended. To avoid
-these issues, we recommend to use an open-ended list of values instead
-of an enumeration unless:
+complete and not intended for extension. This means, extending the list of values of
+`enum` is considered an incompatible change, and needs to be aligned with all consumers
+as other incompatible changes.
+
+To avoid these issues, we recommend to use `enum` only if
 
 1. the API has full control of the enumeration values, i.e. the list of values
   does not depend on any external tool or interface, and
 2. the list of values is complete with respect to any thinkable and unthinkable
   future feature.
 
-To specify an open-ended list of values via the {x-extensible-enum} property as follows:
+In all other cases, where additional values are imaginable our recommendation is this:
+
+* Use `examples` with the list of currently known values
+* Put "Extensible enum" in the description.
+
+This indicates that currently these listed values will be possible, but
+consumers need to be aware that this list can be extended without notice
+(see below for details).
 
 [source,yaml]
 ----
-delivery_methods:
+delivery_method:
   type: string
-  x-extensible-enum:
+  examples:
     - PARCEL
     - LETTER
     - EMAIL
+  description: Extensible enum. The chosen delivery method of the invoice.
 ----
 
-*Note:* {x-extensible-enum} is a proprietary extension of the JSON Schema standard that 
-is e.g. visible via Swagger UI, but ignored by most other tools.
-
 See <<240>> about enum value naming conventions.
 
-Note, {x-extensible-enum} is a different concept than JSON schema `examples` which is 
-just a list of a few example values, whereas {x-extensible-enum} defines all valid 
-values (for a specific API and service version) and has the advantage of an extensible 
-full type-range specification that is validated by the service.
+**Important**:
+
+* API consumers must be prepared for the fact that also other values can be returned
+  with server responses (or be contained in consumed events), and implement a
+  fallback / default behavior for unknown new values, see  <<108>>.
+* API owners must take care to extend these extensible enums in a compatible way, i.e.
+  not changing the semantics of already existing / documented values.
+* For uses in input, API implementations should validate the values and only accept
+  values listed in `examples`. The list should not be reduced (that would be an incompatible change).
+* Before additional values are accepted or returned, API owners should extend
+  the `examples` list, see <<107>>.
+
+(Note that the last 3 bullet points do not apply for uses of `examples` _without_ the
+ "Extensible enum" marker in the description – here any value at all needs
+ to be expected.)
+
+=== Historic Note
+
+Previously (until May 2025), this guideline recommended our own proprietary
+{x-extensible-enum} JSON schema extension here, with a similar semantic:
+
+> This is the *complete* list of values *currently* possible, but consumers must be
+prepared for other values in the future.
+
+Until all existing APIs using this have been updated, API providers and consumers
+still need to follow the rules under "Important" above for these cases.
+
+This completeness semantic would in theory allow some validation by
+intermediaries (but that was rarely implemented).
+It was visible in a few tools (e.g. Swagger UI), but ignored by most others.
+
+An open-ended list of values was specified as follows:
+
+[source,yaml]
+----
+delivery_methods:
+  type: string
+  x-extensible-enum:
+    - PARCEL
+    - LETTER
+    - EMAIL
+  description: The chosen delivery method of the invoice.
+----
 
-*Important:* Clients must be prepared for extensions of enums returned with server responses, i.e. 
-must implement a fallback / default behavior to handle unknown new enum values -- see <<108>>. 
-API owners must take care to extend enums in a compatible way that does not change the 
-semantics of already existing enum values, for instance, do not split an old enum value 
-into two new enum values. Services should only extend {x-extensible-enum} ranges, and only accept 
-and return values listed in the API definition, i.e. the API definition needs to be updated first  
-before the service accepts/returns new values -- see also <<107>>. 
+This rule originated in the time before JSON schema and OpenAPI schema
+had the plural `examples` property (OpenAPI schema had singular `example`,
+JSON schema had neither).
 
+We also thought we could have some validations (e.g. by our event bus Nakadi),
+but the Nakadi team instead decided to not validate `x-extensible-enum`,
+and even reject it in schema definitions.

chapters/events.adoc

@@ -953,23 +953,29 @@ maintain compatibility as they will not be in a position to serve
 versioned media types on demand.
 
 For event schema, these are considered backward compatible changes, as
-seen by consumers -
+seen by consumers:
 
 * Adding new optional fields to JSON objects.
 * Changing the order of fields (field order in objects is arbitrary).
 * Changing the order of values with same type in an array.
-* Removing optional fields.
+* Removing optional fields (or rather, stop sending values for the optional
+  field – actually removing it from the schema is not allowed, as otherwise
+  it could be added back with an incompatible type).
 * Removing an individual value from an enumeration.
-* Adding new value to a {x-extensible-enum} field (see <<112, rule 112>> and <<108, rule 108>>).
+* Adding new values to extensible enum fields (see <<112, rule 112>> and <<108, rule 108>>).
 
 These are considered backwards-incompatible changes, as seen by
-consumers -
+consumers:
 
 * Removing required fields from JSON objects.
 * Changing the default value of a field.
 * Changing the type of a field, object, enum or array.
 * Changing the order of values with different type in an array (also
 known as a tuple).
-* Adding a new optional field to redefine the meaning of an existing
-field (also known as a co-occurrence constraint).
-* Adding a value to an enumeration. Instead, you <<112>>. 
+* Adding a new optional field to redefine the meaning of a (previously)
+  existing field (also known as a co-occurrence constraint).
+* Adding a value to an `enum` enumeration. Instead, you <<112>>.
+
+When an incompatible change is required, it needs to be aligned with all consumers.
+Only start sending values not covered by the previous schema after all consumers
+are prepared to consume the new values.

chapters/json-guidelines.adoc

@@ -136,13 +136,15 @@ as if it came from the same hand.
 
 Enumerations should be represented as `string` typed OpenAPI definitions of
 request parameters or model properties.
-Enum values (using `enum` or {x-extensible-enum}) need to consistently use
-the upper-snake case format, e.g. `VALUE` or `YET_ANOTHER_VALUE`.
-This approach allows to clearly distinguish values from properties or other elements.
+Enum values (using `enum`, {x-extensible-enum} or extensible enums using `examples`)
+need to consistently use the upper-snake case format, e.g. `VALUE`
+or `YET_ANOTHER_VALUE`.
+This approach allows to clearly distinguish enum values from properties or
+other elements.
 
 **Exception:** This rule does not apply for case sensitive values sourced from outside
-API definition scope, e.g. for language codes from {ISO-639-1}[ISO 639-1], or when
-declaring possible values for a <<137,rule 137>> [`sort` parameter].
+of the API definition scope, e.g. for language codes from {ISO-639-1}[ISO 639-1],
+or when declaring possible values for a <<137,rule 137>> [`sort` parameter].
 
 
 [#235]