Microservices by Martin Fowler

In the last 6 months or so, I’ve become a huge fan of Microservices. I love the concept because Microservices are a fantastic extension of domain driven design. One of the core behaviors of Microservices is that no other process is allowed to talk to the data it contains directly (in a database, for example). Any process that needs access to this data, needs to talk to the Microservice. This is exactly what a business entity (aggregate or not) in a domain model is doing. This is what encapsulation in object orientation is all about. To be more specific, a typical domain object has behavior and data, as you know. A domain object will not (and should not) let any other object or process access its data until it passes through its behavior(s). That’s why domain objects (entities) have clearly defined interfaces to talk to.

Now, if you scale this behavior up into components that run in their own processes, these microservices are doing the same thing but from a component point of view: “No other object running in another process is allowed to access another process’s data unless it talks to the behavior of this other component, the other microservice”. This has been my observation and I really have become to like the concept of Microservices. Microservices are a fantastic way of building cloud-based software and with the right PaaS solution, you can create Microservices on premise and simply deploy to a PaaS that also runs on premise or in a public cloud such as Cloud Foundry or AWS, for example.

I’m a huge fan of Martin Fowler. It is not “official” until Martin Fowler has written or spoken about any subject in the Software Industry. Anyway, here is Martin Fowler’s awesome 25 minute explanation of Microservices.

Presenting “Object Persistence in C#” in Sacramento, CA, March 25th, 2015

Wednesday, March 25th, 20015, I will be presenting “Object Persistence in C#” at the Sacramento .NET User Group (SAC.NET) at the Microsoft Office at 1415 L Street, Suite 200, Sacramento, CA 95814 starting at 6:00 pm. Maria Martinez, Co-Organizer, Sacramento .NET User Group, was kind enough in helping to get this organized. Thank you Maria. I will see you there.

Presenting “Object Persistence in C#” at Bay.NET User Group on April 9th, 2015

I will be presenting “Object Persistence in C#” at the Bay.NET user group at the Berkeley City College located at Room 451A, 2050 Center Street, Berkeley, CA from 6:15 pm – 9:00 pm.

Deborah Kurata, Co-Organizer, East Bay Chapter Leader, was kind enough in helping to get this organized. Thank you Deborah.

I will see you there.

Object Persistence, Part 5 – Video – Redis Provider

In part 4 of this series, I went through the entire Visual Studio solution and also showed the db4o object database provider.

In part 5, I’ll show you how to store your Plan Old C# Objects (POCO) into Redis using the Redis Cloud at redislabs.com service. We will build the Redis Provider and take it out for a spin storing our new domain objects.

You can download the source code at GitHub.

Object Persistence, Part 4 – Video

In part 3 of my Object Persistence series, I introduced the complete Visual Studio 2012 source code.

This is part 4 of my Object Persistence series. I created a detailed video that goes thru all the parts in the solution. Enjoy! Make sure you continue with part 5 where we build a Redis persistence provider.

Watch the video walkthrough on my YouTube channel:

Object Persistence, Part 2

It has been several years (six years to be precise) since I published my article “What is Object Persistence”. I have received great feedback since then from the blog post and from presentations I gave about object persistence. However, the challenge of using an appropriate object persistence mechanism still exists today. Fortunately, there are even more ways to store your objects nowadays when compared to 2008. With the great opportunities that cloud computing offers, object persistence gets even more exciting.

So, I decided to publish a follow-up blog post about object persistence. In addition, I will also provide working C# code so that you can try it out yourself. Since object persistence is such an important piece of a software architecture and the depth of technical information about it can be overwhelming, I may have to spread out my thoughts and example source code over additional blog posts.

Most of the example source code I will be providing is coming straight from production systems I have built over the years with .Net and C#. Specifically, I will be providing persistence providers that you can use in your own systems or at least provide you with a huge head start. Some of the source code is changed to accommodate the example better but the provider pattern and the overall design is identical. The source code will be in C# and I will be using a .Net feature that has been available since .NET 2.0 – The Provider Pattern.

So, having said all this, I can pickup where I have left off with my first blog post. Say, you are familiar with the challenges of finding the right object persistence for your project. Let’s also assume you know “where” you want to store your objects in. If you are not familiar of what object persistence is, please take a look at my previous blog post “What is Object Persistence”.

Let’s start with a straight forward object persistence that will help you see the bigger picture and not get lost in the actual details of “how” to store the objects. At least for now. Let’s start with storing our objects in an object database named db4o. The reason why I want to start out with db4o is because it actually is the easiest way in .Net to persist your objects. I would argue that object databases can be used in at least 90% of .Net projects developed today. db4o has another advantage in that it can also run entirely in memory alone which is great for unit testing your persistence. In addition, db4o has such an extremely low learning curve that you will be up and running in no-time. Of course, the beauty of using a provider model is that you can do entirely different persistence implementations of the same domain model. So, you can use a different object database such as VelocityDB, for example.

Later on, I will show you how to store the same C# objects in different ways including the in memory version of db4o, a NoSQL solution such as Redis, which is a key-value store, and to round it out, a typical SQL storage such as SQL Server.

From an architecture point of view, it is very, very, important that our domain model has absolutely no clue about persistence. Our domain model will have no references to any persistence assemblies. Our domain model will be a lone assembly with no references to any service, interfaces, UI, and especially any persistence technologies. This is important because we want our domain model to be maintainable over time. You want your domain model to be independent from any other building blocks of your architecture because it will reflect your business domain and processes. This will make your entire system much easier to maintain and therefore much easier to react to requirements changes.

The Provider Model

The great thing about using the provider model is that the entire implementation is done inside a provider. Your entire source code on “how” to persist your objects is inside the specific provider. If you are not familiar with the provider model, please take a look at these resources to get familiar with it.

Microsoft ASP.NET 2.0 Providers: Introduction

Develop Provider-based Features of Your Application

Besides the introduction of generics in .NET 2, the provider model was in my opinion one of the most powerful features introduced. The .NET framework has been using the provider model internally ever since, all the way to the latest version of .NET 4.5. Here are some examples, where Microsoft is using the provider model:

Membership
Role management
Site map
Profile
Session state
Web events
Web Parts personalization
Protected configuration

and these are just a few of the current providers that ship with the framework. You can even build providers based on certain features of your system, for example. See the link above.

One of the great features of the provider model is that the framework will automatically load your persistence provider based on configuration information. This means that you do not even need any assembly references to your provider, the .NET framework will take care of the discovery, loading, and instanciation for you. This offers a truly decoupled implementation, a true plug-play mechanism out of the box. How cool is that?

Please keep in mind that this has nothing to do with the Repository pattern. I have used the provider model pattern for persistence for many years now and the Repository pattern does not come close to what the provider pattern can do for you. The Repository pattern violates the domain model encapsulation because the domain model is now aware of some sort of persistence idea even if the Repository is exposed via IRepository, for example. That is a big no no in my book because your goal should be to create something that is easy to maintain over a very long time.

If you are building a professional software solution, you should go with the provider pattern for abstracting persistence.

This took longer than I thought but I believe that I needed to set the stage first before we can continue. In the next blog post, we’ll get our hands dirty and start writing code and you will see how it can be done.

Continue with part 3, Object Persistence

What is Object Persistence

UPDATED: Object Persistence, Part 2 (please see my updated blog post for more information)

In the last few decades, there have been different ways of creating software solutions from procedural top-down development to object oriented and service oriented solutions. They all have their place and were created based on different requirements, budgets, and skill sets. There are always different ways on how a software solution can be created and implemented. One way of implementing a software solution is to use Object Oriented Software Development (OOSD) and its many advantages. If you are on a project that does not necessarily take advantage of OOSD, that’s fine, too. Not all projects may require a full OOSD solution. In addition, you can always refactor parts of a solution to take advantage of OOSD techniques if time and budget permit or if new business requirements are so that it calls for a OOSD solution.

Object Oriented Software Development is a powerful and proven technique to create software solutions that mimic a network of real life processes, tasks, and business rules of a domain. Most importantly, as you design and develop your solution, you start thinking in a domain specific language. You acquire a deeper understanding of the business. Domain Driven Design (DDD) is a technique within OOSD that allows you to get very intimate with the business and requirements at hand. Eric Evans, author of Domain Driven Design, explains in great detail on how this can be accomplished. I highly recommend his book if you are interested in this exciting technique.

The premise of domain-driven design is two-fold:

* For most software projects, the primary focus should be on the domain and domain logic; and

* Complex domain designs should be based on a model.

Domain-driven design is not a technology or a methodology. It is a way of thinking and a set of priorities, aimed at accelerating software projects that have to deal with complicated domains.

To accomplish that goal, teams need an extensive set of design practices, techniques and principles.

The Challenge of Complexity

Of course, many things can put a project off course, bureaucracy, unclear objectives, lack of resources, to name a few, but it is the approach to design that largely determines how complex software can become. When complexity gets out of hand, the software can no longer be understood well enough to be easily changed or extended. By contrast, a good design can make opportunities out of those complex features.

Some of these design factors are technological, and a great deal of effort has gone into the design of networks, databases, and other technical dimension of software. Books have been written about how to solve these problems. Developers have cultivated their skills.

Yet the most significant complexity of many applications is not technical. It is in the domain itself, the activity or business of the user. When this domain complexity is not dealt with in the design, it won’t matter that the infrastructural technology is well-conceived. A successful design must systematically deal with this central aspect of the software.

Source: http://domaindrivendesign.org

Many books have been written about OOSD and I encourage you to dig deeper if you are interested in learning specifics. Besides Eric Evans’ book about Domain Driven Design and Domain Driven Design Quickly, I have also been recommending The Object Primer, by Scott Ambler. I highly recommend his Object Primer and urge you to find some time for it.

What is Object Persistence?

If you are on a project that takes advantage of OOSD techniques, chances are that you also had to worry about storing your domain objects at some point.

When you are dealing with a domain model, business objects working in harmony with other business objects, you will need to deal with Object Persistence sooner rather than later. Object Persistence can be described as a running domain model to survive power failures or power spikes so that no data is lost. Some describe Object Persistence as serializing objects out to the disk and retrieving them later. Serializing an object usually means to store the values of public properties of an object. De-serializing would be just the opposite and retrieve the values and reconstruct the original object back to its original state. Others describe Object Persistence as storing objects into a RDBMS where you will encounter the difficulties of the impedance mismatch between the three dimensional object worlds and flat, two dimensional relational databases equivalent of trying to fit a square into a round hole.

In short, Object Persistence is storing the data, the behavior, and the relationships with other objects for later reuse to output valuable information.

A typical Object Persistence can easily consume 50% – 60% of a project’s resources and time if not handled correctly. This time is not just the initial time when a system is being developed. More importantly, the future maintenance of a system is the most expensive part of a system during its lifetime until retirement. If Object Persistence takes up such a big part, shouldn’t Object Persistence deserve a lot more attention?

There are several ways on how Object Persistence can be done. I will go into some details in future posts. Until then, think about the time and productivity that can be gained through effective Object Persistence and not just during prototyping but full blown production systems. How much time do you usually spend dealing with creating a database schema, creating some layer to store “stuff” into tables, and how much time you spend updating all these parts until the system goes into production. Once in production, how much time do you spend making changes as requirements change? How much plumbing work do you do vs. actually solving the domain problems?

In future blogs, I will post about Object Persistence and why this is so important in software development projects that do utilize OOSD today. I will take ideas from my upcoming book The Abundance of Object Persistence to highlight the importance of this crucial part in system architecture not only from a technical perspective but also from a business point of view. Please feel free to provide any comments or suggestions. I’m looking forward to your feedback.