Introducing EventCatalog

I would like to introduce you to my next Open Source tool called EventCatalog.

EventCatalog has been developed to help you become more productive through documentation with Event-Driven Architectures.

Motivation

Over the past few years we have seen event-driven architecture (EDA) adoption increase.

In 2021 InfoQ released the "Architecture and Design 2021 Graph" that shows us adoption phases of technology.

https://www.infoq.com/articles/architecture-trends-2021/

Towards the right of the graph we can see the late majority of accepted technologies. Event architecture patterns such as (Event Sourcing, EDA, CQRS) fit into this category after the majority of society has adopted these innovations.

We can also see that other key parts of the EDA landscape are being adopted (Serverless, Distributed Systems, AsyncAPI, Design for Observability).

We have witnessed over the past few years serverless technology is reducing the barrier to entry into EDA and the serverless adoption increasing.

It's clear that many patterns and technologies have already been established within the EDA landscape, but after experience, reading and talking to people I believe there are many areas that still need to be solved.

Specifications, Documentation and Observability

Many people I speak to about EDA often come back to observability issues and the need for some form of documentation surrounding events, producers, consumers, schemas and more...

Luckily we have open source communities and standards growing (AsyncAPI, CloudEvents, Serverless Workflow) to help us document our EDA architecture.

As these standards continue to gain traction and adoption we continue to see the rise of organisations that need help documenting, discovering and observing their Event Architectures.

So where does EventCatalog fit into all of this? And how did it start?

The Problem

In 2021 I released a couple of Open Source projects around Amazon EventBridge (EventBridge Atlas and EventBridge Canon) which seemed to get a fair amount of interest (which I found interesting). The problems I was facing with documentation, discovery and tooling was also being felt by others within the community.

These projects got interest from the AsyncAPI community and I was offered to join Postman open technology team working with the AsyncAPI community. Which I accepted!

This new role has allowed me to spend a lot of time consuming, learning and exploring the Event Architecture domain.

As I engage with others and consume content I’m discovering our communities are developing standards around EDA and tooling that helps us architect, code and create our systems...

But I feel discovery and documentation is still lagging behind…

James Urquhart describes in his book "Flow Architectures: The Future of Streaming and Event-Driven Integration" a world where real-time data is more accessible. Data can cross organisational boundaries through real-time streams and how event-driven architecture will form a base for that.

For whatever reason the idea of Flow, real-time data, events, architecture all resonates with me. I can actually see the future here...

What's really interesting to me is that James shows a Wardley Map which documents the landscape of technology towards the idea of flow.

Flow Wardley Map

The areas of the map on the left are unknown whilst areas on the right are more known.

I believe this is true for our current state of EDA. For example we have many infrastructure and cloud based solutions to help us with EDA but discovery (on the left) is still unknown but is starting to play a key part of our EDA systems.

At the moment many brokers have tools like Schema registries which help us understand the shape/payload of our messages/events but I believe it’s not enough…

I believe we can go beyond the schema registry…

How can we improve discovery/documentation?

Over the past decade we have seen some huge investments in web development tooling and frameworks (NextJS, Netlify, Remix, React etc…) which allows us to rapidly create applications.

The best thing about these high performing frameworks is the developer experience.

Which got me thinking...

What if we had a tool that focuses on the developer experience and allows us to document, discover and explore our event architectures?

For adoption I believe it would need to meet some basic criteria:

• Documentation has to be minimal effort
• Developer experience has to be important (think NextJS, Remix etc)
• Content creation needs to be simple.
• Plugin Architecture (fit into existing architectures)
• Technology Agnostic. Allow users to benefit from any EDA implementation.
• Integrate with stanards (AsyncAPI, CloudEvents etc)
• Generate docs from other systems (registries, topics, etc)

Over time these thoughts consumed me...

The more I was reading and connecting with people the more I felt a tool like this could really benefit organisations and our community.

So I decided to start building EventCatalog and share its progress in public.

Over the past couple of months EventCatalog has had more engagement than I ever thought it would.

It confirms that discovery/documentation of EDA is a real problem that we need to solve.

If you are into Event Architecture and want an Open Source tool to help:

🔎 Discover Events
📄 Document Events
📊 Visualise Consumers/Producers
🧑‍💻 Schema + Code Examples
And much more...

I'm working on something that you might be interested in 👇🧵

— David Boyne 🚀 (@boyney123) January 5, 2022

So today, I’m happy to release the initial version of EventCatalog.

What's next for EventCatalog

During EventCatalogs development, I had to throw out features/ideas on the backlog to actually get a release out the door. But here some initial thoughts/ideas for the next phases:

• Well defined Plugin API (to help people integrate into their Applications)
• Better integration with Stanards (AsyncAPI, CloudEvents)
• Allow documentation of domains, commands and more
• Versioning of Services (Events supported)
• More visual diagram support
• and much more...

EventCatalog was designed and built for our community. If you wish to get involved free free to check out the project and join us on discord.

How to contribute

Here is just a few ways you can help contribute to EventCatalog:

• Feedback (Bugs & Thoughts)
• Ideas for EventCatalogs future
• Understanding your usecase
• Code Contributions
• Join Discord

Summary

I hope EventCatalog can help you and your teams start to explore, discover and document your event-architectures. I feel this kind of tooling is needed within our EDA community and I hope you find it useful.

If you any feedback you can reach me on Twitter.

Until next time!