The consumers using the schema V3 can process data written by producers using the schema V3 or V2.Ĭonsumers using the new schema can process data written by producers using all previous schemas. Suppose that you have a topic containing three schemas (V1, V2, and V3), V1 is the oldest and V3 is the latest: Compatibility check strategyĬonsumers using the new schema can process data written by producers using the last schema. Optionally, create a new topic and start migrating applications to use the new topic and the new schema, avoiding the need to handle two incompatible versions in the same topic.
#Airmail 2.5.1 compatibility upgrade
Upgrade all producers and consumers to the new schema versions at the same time. Sometimes we also make incompatible changes.įor example, you are modifying a field type from string to int. This application can use ALWAYS_COMPATIBLE to allow different kinds of events co-exist in the same topic. The application requires that those events are always read in the same order.Ĭonsequently, those events need to go in the same Pulsar partition to maintain order. In particular, when developing a data model in an Event Sourcing style, you might have several kinds of events that affect the state of an entity.įor example, for a user entity, there are userCreated, userAddressChanged and userEnquiryReceived events. In some situations, an application needs to store events of several different types in the same Pulsar topic. Suppose that you have a topic containing three schemas (V1, V2, and V3), V1 is the oldest and V3 is the latest:Ĭonsumers using the schema V3 can process data written by producers using the schema V3 or V2.Ĭonsumers using the schema V3 can process data written by producers using the schema V3, V2 or V1.Ĭonsumers using the schema V3 or V2 can process data written by producers using the schema V3.Ĭonsumers using the schema V3, V2 or V1 can process data written by producers using the schema V3.īackward and forward compatible between the schema V3 and V2.īackward and forward compatible among the schema V3, V2, and V1.ĪLWAYS_COMPATIBLE and ALWAYS_INCOMPATIBLEĭisable schema evolution, that is, any schema change is rejected.įor all schema types except Avro and JSON, the default schema compatibility check strategy is ALWAYS_INCOMPATIBLE.įor Avro and JSON, the default schema compatibility check strategy is FULL. Pulsar has 8 schema compatibility check strategies, which are summarized in the following table. The broker uses the checker to check if the SchemaInfo is compatible with the latest schema of the topic by applying its compatibility check strategy.Ĭurrently, the compatibility check strategy is configured at the namespace level and applied to all the topics within that namespace. The broker knows the schema type and locates the schema compatibility checker for that type. The producer/consumer/reader sends its client SchemaInfo to the broker. The schema compatibility checker is one instance per schema type.Ĭurrently, Avro and JSON have their own compatibility checkers, while all the other schema types share the default compatibility checker which disables schema evolution. When a producer/consumer/reader connects to a broker, the broker deploys the schema compatibility checker configured by schemaRegistr圜ompatibilit圜heckers to enforce schema compatibility check. How does Pulsar support schema evolution?
It determines how schema compares old schemas with new schemas in topics.įor more information, see Schema compatibility check strategy.
The answer is Pulsar schema compatibility check strategy. Schema evolution ensures that the downstream consumers can seamlessly handle data encoded with both old schemas and new schemas.
#Airmail 2.5.1 compatibility update
To satisfy new business requirements, you need to update schemas inevitably over time, which is called schema evolution.Īny schema changes affect downstream consumers. Schemas store the details of attributes and types. When a message is consumed by a Pulsar client, the Pulsar client can use the schema version to retrieve the corresponding SchemaInfo and use the correct schema information to deserialize data. The message produced with SchemaInfo is tagged with a schema version. The version is used to manage the schema changes happening within a topic. Pulsar schema is defined in a data structure called SchemaInfo.Įach SchemaInfo stored with a topic has a version. This chapter examines how Pulsar schema evolves and what Pulsar schema compatibility check strategies are. Instead, they undergo evolutions to satisfy new needs.
Normally, schemas do not stay the same over a long period of time.
0 kommentar(er)
