Object Persistence Reference Implementation

I’ve been updating my reference implementation in the last few days. I’m actually using this reference implement in my own projects. You can download the latest version on my GitHub repo.

This is a complete .NET C# reference implementation to help you jump start a service oriented system running in a cloud environment such as Amazon’s EC2 or on-premis clusters.

This reference implementation shows you how to build a client and the server side. The client side is a sample WPF application that communicates via http REST requests using JSON payloads to the service side. Of course, you can use any type of client as long as the client can communicate via http and REST based JSON’s.

The service side is using a Web API 2 service layer that communicates to a central domain model. The service side demonstrates how to handle exceptions and edge cases and how to communicate failure to the client.

The persistence layer demonstrates the extreamly powerful provider pattern to store the domain objects into the following databases:

  1. db4o (an object database)
  2. Redis (a NoSQL database)
  3. SimpleDB (a NoSQL database)
  4. SQL Server (comming soon)

Please note that the entire system has no knowledge on how the objects are stored. All implementation details are in the individual providers listed above. This means that you can switch the persistence provider without having to recompile and therefore switch a running system from one persistence store to another.

I will try to create a sample SQL Server provider soon.

Presenting “Object Persistence in C#” in Sacramento, CA, March 25th, 2015

Wednesday, March 25th, 20015, I will be presenting “Object Persistence in C#” at the Sacramento .NET User Group (SAC.NET) at the Microsoft Office at 1415 L Street, Suite 200, Sacramento, CA 95814 starting at 6:00 pm. Maria Martinez, Co-Organizer, Sacramento .NET User Group, was kind enough in helping to get this organized. Thank you Maria. I will see you there.

Presenting “Object Persistence in C#” at Bay.NET User Group on April 9th, 2015

I will be presenting “Object Persistence in C#” at the Bay.NET user group at the Berkeley City College located at Room 451A, 2050 Center Street, Berkeley, CA from 6:15 pm – 9:00 pm.

Deborah Kurata, Co-Organizer, East Bay Chapter Leader, was kind enough in helping to get this organized. Thank you Deborah.

I will see you there.

Object Persistence, Part 5 – Video – Redis Provider

In part 4 of this series, I went through the entire Visual Studio solution and also showed the db4o object database provider.

In part 5, I’ll show you how to store your Plan Old C# Objects (POCO) into Redis using the Redis Cloud at redislabs.com service. We will build the Redis Provider and take it out for a spin storing our new domain objects.

You can download the source code at GitHub.

Object Persistence, Part 4 – Video

In part 3 of my Object Persistence series, I introduced the complete Visual Studio 2012 source code.

This is part 4 of my Object Persistence series. I created a detailed video that goes thru all the parts in the solution. Enjoy! Make sure you continue with part 5 where we build a Redis persistence provider.

Watch the video walkthrough on my YouTube channel:

Object Persistence, Part 3 – Source Code

In part 2 of my Object Persistence series, I’ve touched on the issues that still exist today.

In part 3, I’ve published a complete sample Visual Studio 2012 solution on GitHub that demonstrates object persistence using a db4o persistence provider. Over time, I will add additional sample persistence providers for Redis, SQL Server, and possibly a NoSQL provider such as SimpleDB (one of Amazon’s great NoSQL databases).

This sample solution includes complete server side and client side layers. The Server side runs as a REST based Web API 2 service. The server portion also includes a simple domain model and, of course, the persistence provider and how it is implemented. I will update the solution over time, expand the domain model, UI, etc. as required.

The client side is a WPF application that consumes the REST service. The payload to and from the REST service is via JSON objects.

ObjectPersistencePart3_WPF

I hope you like it. You can use this sample solution as a template to start simple or very complex software solutions. This solution can easily be taken and split across different nodes in a cluster of Amazon AWS EC2 instances, for example. However, for a cloud based solution, your persistence would have to support certain features. I will go into details when I add the Redis persistence provider.

Instead of writing a very long blog post, I will post a screen cast video and go through the solution. I think this will make more sense and you have a chance to go through the source code with me. So, go ahead and get the latest version from GitHub and start playing with it.

Object Persistence, Part 2

It has been several years (six years to be precise) since I published my article “What is Object Persistence”. I have received great feedback since then from the blog post and from presentations I gave about object persistence. However, the challenge of using an appropriate object persistence mechanism still exists today. Fortunately, there are even more ways to store your objects nowadays when compared to 2008. With the great opportunities that cloud computing offers, object persistence gets even more exciting.

So, I decided to publish a follow-up blog post about object persistence. In addition, I will also provide working C# code so that you can try it out yourself. Since object persistence is such an important piece of a software architecture and the depth of technical information about it can be overwhelming, I may have to spread out my thoughts and example source code over additional blog posts.

Most of the example source code I will be providing is coming straight from production systems I have built over the years with .Net and C#. Specifically, I will be providing persistence providers that you can use in your own systems or at least provide you with a huge head start. Some of the source code is changed to accommodate the example better but the provider pattern and the overall design is identical. The source code will be in C# and I will be using a .Net feature that has been available since .NET 2.0 – The Provider Pattern.

So, having said all this, I can pickup where I have left off with my first blog post. Say, you are familiar with the challenges of finding the right object persistence for your project. Let’s also assume you know “where” you want to store your objects in. If you are not familiar of what object persistence is, please take a look at my previous blog post “What is Object Persistence”.

Let’s start with a straight forward object persistence that will help you see the bigger picture and not get lost in the actual details of “how” to store the objects. At least for now. Let’s start with storing our objects in an object database named db4o. The reason why I want to start out with db4o is because it actually is the easiest way in .Net to persist your objects. I would argue that object databases can be used in at least 90% of .Net projects developed today. db4o has another advantage in that it can also run entirely in memory alone which is great for unit testing your persistence. In addition, db4o has such an extremely low learning curve that you will be up and running in no-time. Of course, the beauty of using a provider model is that you can do entirely different persistence implementations of the same domain model. So, you can use a different object database such as VelocityDB, for example.

Later on, I will show you how to store the same C# objects in different ways including the in memory version of db4o, a NoSQL solution such as Redis, which is a key-value store, and to round it out, a typical SQL storage such as SQL Server.

From an architecture point of view, it is very, very, important that our domain model has absolutely no clue about persistence. Our domain model will have no references to any persistence assemblies. Our domain model will be a lone assembly with no references to any service, interfaces, UI, and especially any persistence technologies. This is important because we want our domain model to be maintainable over time. You want your domain model to be independent from any other building blocks of your architecture because it will reflect your business domain and processes. This will make your entire system much easier to maintain and therefore much easier to react to requirements changes.

The Provider Model

The great thing about using the provider model is that the entire implementation is done inside a provider. Your entire source code on “how” to persist your objects is inside the specific provider. If you are not familiar with the provider model, please take a look at these resources to get familiar with it.

Microsoft ASP.NET 2.0 Providers: Introduction

Develop Provider-based Features of Your Application

Besides the introduction of generics in .NET 2, the provider model was in my opinion one of the most powerful features introduced. The .NET framework has been using the provider model internally ever since, all the way to the latest version of .NET 4.5. Here are some examples, where Microsoft is using the provider model:

Membership
Role management
Site map
Profile
Session state
Web events
Web Parts personalization
Protected configuration

and these are just a few of the current providers that ship with the framework. You can even build providers based on certain features of your system, for example. See the link above.

One of the great features of the provider model is that the framework will automatically load your persistence provider based on configuration information. This means that you do not even need any assembly references to your provider, the .NET framework will take care of the discovery, loading, and instanciation for you. This offers a truly decoupled implementation, a true plug-play mechanism out of the box. How cool is that?

Please keep in mind that this has nothing to do with the Repository pattern. I have used the provider model pattern for persistence for many years now and the Repository pattern does not come close to what the provider pattern can do for you. The Repository pattern violates the domain model encapsulation because the domain model is now aware of some sort of persistence idea even if the Repository is exposed via IRepository, for example. That is a big no no in my book because your goal should be to create something that is easy to maintain over a very long time.

If you are building a professional software solution, you should go with the provider pattern for abstracting persistence.

This took longer than I thought but I believe that I needed to set the stage first before we can continue. In the next blog post, we’ll get our hands dirty and start writing code and you will see how it can be done.

Continue with part 3, Object Persistence

Benefits of Object Persistence

When Object Persistence is done right, there can be some great rewards in the areas of Productivity, Maintainability, and Cost Reduction. Some areas directly affect other areas. For example, if one can maintain an application fairly easily, this would result in a higher productivity. There are fine balances between these areas.

 
Productivity

Productivity in software development can be measured in how fast a developer can complete a given work item with the assumption that the quality of the software is not compromised. How much time does it take to get things done without any compromises? Because most projects have a limited amount of time to complete, being productive impacts Cost Reduction to create and maintain the application as well as a better chance for future funding. Being productive should mean that one can get the work done and still deliver high quality software, with a high degree of reliability, and a fair amount of maintainability. Compromises in any one of these areas will result in loss of productivity either in the short-term or long-term of the product life-cycle.

The Object Persistence can be designed in such a way that the system is not locked into a particular database, for example. Because there is no database vendor lock-in, you can gain productivity by simply abstracting storage specific features. For example, if you decide to use an Object Relational Mapping Tool (ORM), you should let the ORM tool worry about creating and maintaining the database schemas for any database it supports based on your Object Model. This can buy you a tremendous amount of productivity because you are delegating this huge maintenance burden to the ORM tool.

  • Productivity in software development can be gained through abstraction and automation.

Maintainability

Depending how Object Persistence is done, it usually allows for easier maintainability when requirements change. The fewer changes that need to be made to a system, the easier it is to maintain. Because of abstracting the object persistence, there is a high possibility that a system can be created that is easy to maintain. Because you no longer have to make changes to a database schema or data access layer, you can concentrate on making the changes usually only in one place: the object model and sometimes to the output such as the user interface or a new report

How many times have you read somebody else’s code and trying to figure out what the heck it is trying to do? Hard to read code, or Spaghetti code, is hard to maintain. In general, the fewer lines of code to maintain, the better. Because of the abstraction of the Object Persistence, much fewer lines of code are necessary. A simple method call such as the Save() method is all it’s needed. This makes the code much easier to read and to maintain.

By eliminating the database schema, eliminating a data access layer, and any data or value objects, we are eliminating several points of failures. By reducing point of failures, the system becomes immediately easier to maintain and more reliable.

The easier it is to maintain an application, the less time is required to train somebody else to learn it. Easy to understand code is easy to support no matter who will need to take over the code.

Maintainability is probably the biggest impact of an application throughout its lifetime much more so than the advantages of reusability in Object Orientation.

Cost Reduction

Studies have shown that in the lifetime of an application, 10% of the time is used to create the application, and 90% is used to maintain it until it is retired. If we can reduce the amount of time needed to maintain the application, we can reduce the amount of money spent to keep it alive. We are trying to work smarter using the right tools instead of harder using no tools or unnecessary procedures.

Being more productive has an immediate impact on cost. When a developer can implement a change quicker, the cost of maintaining an application is reduced. The time saved can be used for other duties or other projects. Many times, being more productive can allow for future projects to be approved vs. projects that won’t make the approval because the timelines would not permit to do so otherwise.

Today, labor cost is one of the most expensive liabilities a company can have. On the other hand, the workforce is also the most important asset a company has. There was a time where hardware was the most expensive part in an IT department. Hardware and tools are now extremely inexpensive today. It is mass produced and no longer a major factor in software development.

Purchasing third party component libraries can reduce the time and effort to complete an application. Some of these tools such as ORM tools can seem expensive initially, but; in the long term, you cannot afford not buying them. It is far more costly to write these components or tools on your own than to purchase them.

New Demo

I have found that quite a few people are using my presentation demo as a guide to start new projects. This was not my intention but rather a very simple example on how you could separate the layers and use db4o.

I realize that it is easier to learn by example especially on topics such as design patterns, persistence, layering a system, etc. So, I decided to start working on a new demo. This new demo will show you how you can effectively layer your system, how you can separate your persistence, how you unit test, implement several design patterns etc.

It will definitely help you to get a major head start when starting a new project. Please let me know if you have any ideas or special cases you would like to consider for the demo.