Communication Patterns: Integration

Last week, we explored drawing boundaries around software parts. What is decoupling, why does it matter and what are some principles for attempting it? This week, we explore patterns and principles for forming new relationships between the (decoupled) parts.

Interestingly, the patterns we rely on today haven’t changed much over time – despite remarkable change in the technology tools that implement them. Like most systems architecture practices, the skills needed to apply them are (somewhat) technology agnostic.

“When Bobby Woolf and I started to document the [integration] patterns 20 years ago, the key technologies for distributed applications were JMS, SOAP, MSMQ, and .NET WCF. Large-scale application integration was done with integration tools and platforms, such as IBM WebSphere MQ, TIBCO, WebMethods (now Software AG), or Microsoft BizTalk. Although technology has evolved, many of these products still form the backbone of modern enterprises.

Cloud platforms and deployment automation have laid the foundation for a new generation of distributed systems: microservices and serverless architectures. Those applications rely on a smooth interconnect between components, giving rise to Service Meshes, Serverless Orchestrators, and Event Buses. Amazingly, we find the same integration patterns in those systems!”

– Gregor Hohpe

Patterns enable communication between software parts without tight coupling. Inherent in pattern design is building the necessary mechanisms, like message queues, Kafka streams and endpoints.

We design:

  1. Sources of information. What information needs to be consumed by other parts? Parts become sources and consumers (or both).
  2. When information is shared. Not everything that happens inside a part needs to be shouted to the world. When does matterful change happen? (Eventstorming can help design this.)
  3. The flow of information. How will information be carried, stored and/or retrieved between parts?
  4. The structure of information. Information stored inside a part, for example, in a MySQL database, won’t be understandable to a more semantic consumer, like a React front-end application. Patterns like canonical data modeling create a ubiquitous structure for information.
  5. Orchestration. Software hosts and manages and cleans up and otherwise orchestrates the relationships between software parts. Software also tests the system in action.

There is no Miracle Pattern that applies to all circumstances. As we move from software to systems (of software), we need two things most of all: experience and sound judgement. It’s tricky to discern which patterns will work best in each circumstance. We don’t (necessarily) want to do what Netflix does, we want to create patterns that enable emergence: parts together accomplish something valuable that none of the parts could do alone.

Experience lessens our desire for control and increases our ability to create malleable architectures. We discern what matters most (clean, helpful communication) and what matters least (despite cultural imperatives, 99.999% is often indistinguishable from 99.9% in a well-designed system). By learning from the experience of others, we avoid building a distributed monolith in the cloud.

All this to say, books and podcasts are helpful. Here are resources to begin exploring. But playing with patterns is more helpful.

As always, we’d love to hear any recommendations you have!

Enterprise Integration Patterns: Designing, Building and Deploying Messaging Solutions

by Gregor Hohpe & Bobby Woolf

This book has been around for decades. Yet, it’s still relevant in critical ways. It’s a good place to start thinking about patterns, which can then be adapted to more-modern situations. You can also  explore them here (if you don’t mind the old school webpage design).

Enterprise Integration Patterns 2

by Gregor Hohpe

This talk at YOW! 2017 explores Enterprise Integration Patterns (see above) … what is next? The author focuses on the thought processes behind patterns. A highly-recommended companion to reading the book.

Grow Your Business Through Enterprise Integration, a Live Interview w/ a Panel of Experts

hosted by WBSRocks Podcast

This podcast is from a decidedly business perspective. When considering pattern changes, it helps to hear the language people use to describe them. We miss a lot when we look at solutions strictly through the lens of technology implementation.

Data Integration: EAI Patterns (Beginner)

taught by Ron Johnson for Skillsoft

If you want to get your hands dirty with some hard-core implementation of a few basics. Fair warning: he says “applications” a lot.

Excessive complexity is nature’s punishment for organizations who are unable to make decisions.

Gregor Hohpe

Writing as Thinking

and learning and leading

January 19 – March 16, 2023

Thursdays for one hour
at Noon ET, 5pm GMT, 6pm CET, 9am PT

Writing as Thinking is an eight-week course for technologists — introducing writing practices for thinking, learning and leading.

Well-reasoned writing transforms disparate thoughts into meaningful action. In this course, you’ll practice writing in ways that help you:

cultivate self awareness
strengthen your recommendations
learn from other people’s expertise
effectively lead knowledge work

Learn more about this course.

The course fee is $499.
The initial (beta) course is on sale for $249. 

Contact Mentrix

Get Updates

From Software to Systems

WELCOME,

Enjoy luxury, exclusivity and discretion

NOW TREAT YOURSELF!

Get 25% Off & Free Shipping On Your First Order. Enter Code WELL25SPE