5 open source standards you should know when building event-driven architectures

Event-driven architectures are gaining popularity as they become increasingly accessible. Cloud providers like AWS and Microsoft are offering ready-made services and solutions, while companies like Confluent continue to expand the Kafka ecosystem.

We are also seeing investment and innovation in this space from companies like CloudFlare, Inngest, restate, and trigger.dev who I believe are lowering the barrier to event-driven architectures.

As the barrier to event-driven architecture lowers, it’s important to keep up to date with open source standards and specifications that can help you and your teams. These standards can help with compatibility, interoperability, and make your systems more adaptable.

Recently I have been diving deeper into standards and specifications for event-driven architectures (building open source EventCatalog integrations) and in this blog post I want to give you an high level overview of them with extra resource to dive deeper.

OpenAPI

The OpenAPI specification (previously Swagger specification) is a community-driven open specification that provides a formal (machine readable) standard for describing HTTP APIs.

OpenAPI specification is language agnostic. OpenAPI allows computers and your teams to discover and understand services without requiring the source code, documentation or deeper dives.

OpenAPI Version 1.0.0 was released in 2011 and is still being actively maintained. Over the past 10+ years the OpenAPI specification is widely adopted and used throughout the industry, many companies building APIs tend to use the OpenAPI specification to help document their APIs.

If you are writing APIs and not using the OpenAPI specification, you are missing out.

Resources

AsyncAPI

The AsyncAPI specification is the industry standard for defining asynchronous APIs. Using the AsyncAPI specification you can document messages in/out of your service, channels it may use and agnostic to any technology or protocol out there.

AsyncAPI is great for event-driven architectures as you can document your producers/consumers and any messages they may use. As AsyncAPI is technology agnostic you can use custom bindings to document any broker or technology you are using.

OpenAPI specification does a great job at documenting request/response models, but hard to document 1:N patterns. AsyncAPI was created to solve this problem.

AsyncAPI started as a fork of OpenAPI by Fran Mendez (founder of AsyncAPI). In 2021 AsyncAPI joined the Linux Foundation project and partnered with Postman to help the project. Since 2021 the project community has grown, the tooling around the specification and companies around the world using it including LEGO Group, Walmart, Postman and eBay.

If you are building event-driven architecture or async workloads I highly recommend checking out this specification and understanding how it can help you and your teams.

Resources

CloudEvents

From EDA Visuals - Free book you can download about event-driven architectures

CloudEvents is a specification for describing event data in common formats to provide interoperability across services, platforms and systems. CloudEvents does not enforce what goes into your envelopes (data inside) but just provides a standard to wrap them (the envelop itself).

Many folks using message based architectures are reinventing the wheel, redefining common properties that go into your message. This lack of common structure makes it hard to learn and consume events across brokers/technology and organisation. Lack of standards also means the potential of libraries and tooling uses for these systems becomes harder.

In October 2019 the CloudEvents specification released the 1.0 version of the specification and in January 2024 CloudEvents graduated in the CNCF.

CloudEvents supports many different protocols including (not limited to) AMQP, Avro, HTTP, Nats, MQTT and SDK support including Go, Java, PHP, JavaScript, Ruby and Rust. You can choose two formats of your CloudEvents including structured mode (CloudEvent metadata inside the payload) or binary mode (CloudEvent metadata as headers).

If you are writing event-driven architectures, I highly recommend learning/reading about CloudEvents. Don’t rewrite what is already done for us, use off the shelf standards, and benefit from the eco-system and tooling.

Resources

Arazzo Specification

The Arazzo Specification (from the OpenAPI Initiative) is a community-driven specification that provides a mechanism that can define sequences of calls and their dependencies to be woven together and expressed in the context of delivering a particular outcome or set of outcomes when dealing with API descriptions.

Using the Arazzo specifcation you can start to document API workflows within your organisation and systems, helping machines and humans understand how flows work. An example of this (from my understanding, which is still new!) could be defining and documenting how can payment system works, for example when a user makes payment. You can use this specification to document that workflow, which others can also read and understand.

In 2024 v1 of the specification was announced and since then gained momentum. The project is open source and you can find examples of the website.

Personally I love the idea of flows, I added these to EventCatalog to help people document their event-driven architectures and workflows that come with that. Having a standard we can also use to document these things is great!

Resources

xRegistry

xRegistry is an open-source project that defines an abstract model for managing metadata about resources. It provides a REST-based interface for creating, modifying, deleting, and discovering these resources.

I’m still learning about this myself, but I wanted to share this with you as I feel they have some great ideas and folks working on this.

To help you understand, this is taken directly from the project primer.

While CloudEvents are harmonizing different event structures across event providers and increase interoperability between different brokers and their protocols, they do not address:

Discovery: Where to produce/consume events? What endpoints exist?
Validation: How to validate event structures?
Versioning: How to version metadata describing events – and see which versions exist?
Serialization: How are events serialized? How do their envelope (if any) and formats (XML / JSON Schema) look like?
Extension Discovery: How to identify which events using which specific extensions?

Schema registries can provide an answer to these questions, but just like event brokers, there are multiple registries, such as the Confluent Schema Registry, the Azure Event Hubs Registry or the Apicurio Registry. When an event travels through multiple brokers, its schema has to be introduced to the broker's registry and clients later have to deal with the implementation differences between different registries, for example when trying to validate an event structure. This hinders the interoperability of event brokers CloudEvents had in mind. xRegistry therefore comes in with a similar motivation to CloudEvents: Harmonize another piece of technology in the messaging / eventing ecosystem to increase interoperability and decouple event handling from broker products and protocols.

Even though xRegistry was built with eventing in mind, the concept of this registry specification is far more powerful and can be used for the exchange for any type of message. Take a look at the Possible Use Cases for examples outside the eventing world.

If you want to dive deeper into this space, I recommend checking out the resource links below. I feel there is a lot of synergy between this project and EventCatalog, and I’m excited to see how EventCatalog can use some of these standards emerging.

Resources

•••

Want to learn more about event-driven architectures?

Get the EDA Visuals Book! (For free)

Over 60 bite sized visuals I created to help you learn about event-driven architectures.

•••

Summary

Building distributed message based architectures are becoming more accessible to us. This means the barrier to entry for these types of architectures is lowering. This is great as it gives us more options for off the shelf integrations and services we can use, but most of these tools do not enforce any standards or specifications.

Using standards and specifications give you superpowers.

If you use standards and specifications you can gain access to a treasure trove of tools and communities. Many developers use standards and specifications to build tools that you and your organisation can benefit from.

I hope this short blog post gave you insights into some specifications and inspired you to dive deeper.