Object Persistence Reference Implementation

I’ve been updating my reference implementation in the last few days. I’m actually using this reference implement in my own projects. You can download the latest version on my GitHub repo.

This is a complete .NET C# reference implementation to help you jump start a service oriented system running in a cloud environment such as Amazon’s EC2 or on-premis clusters.

This reference implementation shows you how to build a client and the server side. The client side is a sample WPF application that communicates via http REST requests using JSON payloads to the service side. Of course, you can use any type of client as long as the client can communicate via http and REST based JSON’s.

The service side is using a Web API 2 service layer that communicates to a central domain model. The service side demonstrates how to handle exceptions and edge cases and how to communicate failure to the client.

The persistence layer demonstrates the extreamly powerful provider pattern to store the domain objects into the following databases:

  1. db4o (an object database)
  2. Redis (a NoSQL database)
  3. SimpleDB (a NoSQL database)
  4. SQL Server (comming soon)

Please note that the entire system has no knowledge on how the objects are stored. All implementation details are in the individual providers listed above. This means that you can switch the persistence provider without having to recompile and therefore switch a running system from one persistence store to another.

I will try to create a sample SQL Server provider soon.

Advertisements

appsworld North America 2015 at Moscone Center West, San Francisco

appsworldlogo

I’m a confirmed speaker at the appsworld North America 2015 at Moscone Center West, May 12-13 in San Francisco, CA.  Discover the future of multi-platform apps. See all confirmed speakers. This sure will be an exciting event. I’m still working on my presentation that will include Redis, Amazon AWS, C#, and more. I will see you there.

Presenting “Object Persistence in C#” at Bay.NET User Group on April 9th, 2015

I will be presenting “Object Persistence in C#” at the Bay.NET user group at the Berkeley City College located at Room 451A, 2050 Center Street, Berkeley, CA from 6:15 pm – 9:00 pm.

Deborah Kurata, Co-Organizer, East Bay Chapter Leader, was kind enough in helping to get this organized. Thank you Deborah.

I will see you there.

Object Persistence, Part 5 – Video – Redis Provider

In part 4 of this series, I went through the entire Visual Studio solution and also showed the db4o object database provider.

In part 5, I’ll show you how to store your Plan Old C# Objects (POCO) into Redis using the Redis Cloud at redislabs.com service. We will build the Redis Provider and take it out for a spin storing our new domain objects.

You can download the source code at GitHub.

Object Persistence, Part 4 – Video

In part 3 of my Object Persistence series, I introduced the complete Visual Studio 2012 source code.

This is part 4 of my Object Persistence series. I created a detailed video that goes thru all the parts in the solution. Enjoy! Make sure you continue with part 5 where we build a Redis persistence provider.

Watch the video walkthrough on my YouTube channel:

Object Persistence, Part 3 – Source Code

In part 2 of my Object Persistence series, I’ve touched on the issues that still exist today.

In part 3, I’ve published a complete sample Visual Studio 2012 solution on GitHub that demonstrates object persistence using a db4o persistence provider. Over time, I will add additional sample persistence providers for Redis, SQL Server, and possibly a NoSQL provider such as SimpleDB (one of Amazon’s great NoSQL databases).

This sample solution includes complete server side and client side layers. The Server side runs as a REST based Web API 2 service. The server portion also includes a simple domain model and, of course, the persistence provider and how it is implemented. I will update the solution over time, expand the domain model, UI, etc. as required.

The client side is a WPF application that consumes the REST service. The payload to and from the REST service is via JSON objects.

ObjectPersistencePart3_WPF

I hope you like it. You can use this sample solution as a template to start simple or very complex software solutions. This solution can easily be taken and split across different nodes in a cluster of Amazon AWS EC2 instances, for example. However, for a cloud based solution, your persistence would have to support certain features. I will go into details when I add the Redis persistence provider.

Instead of writing a very long blog post, I will post a screen cast video and go through the solution. I think this will make more sense and you have a chance to go through the source code with me. So, go ahead and get the latest version from GitHub and start playing with it.

Is There a Problem?

 

The fact that you are using objects in your project or product does not mean that the software development process will be any easier. Essentially, you will run into issues that you need to address. In general, some problems will come up before you start designing your solution, some challenges will come up while you are developing your solution, and others issues will need constant attention.

 These issues can be summarized into three categories:

  • How Object Persistence has been done in the past
  • Technical Challenges
  • Political / Social Hurdles

How Object Persistence Has Been Done in the Past

When you ask a developer about a system that he created and then drill-down into the details such as the “data”, many times you will find a “data-centric” thinking in that person even though he is knowledgeable about object-orientation. You will hear that person talk about stored procedures, SQL’s, datasets, even the term entities, etc. Unfortunately, you will discover that the data model is actually driving the entire application. There may or may not be a domain model; but, there will be a database model.

In a domain driven system, the Domain Model must always drive the Data Model.  Never, must the Data Model drive the Domain Model.

The Domain Model dictates when and how the database schema needs to change. If you need to make a design change in your domain model, the database model needs to be updated accordingly so that it can support the domain model. No compromises in the Domain Model must be taking place to support the database model in any way.

The Domain Model should be designed without any regard to object persistence or user interface exposure.

Think of it like this: Design your domain model as if you have unlimited storage space available and all the horsepower in the world to support your object model. Concentrate on solving the domain problems. Concentrate on solving the use cases. Concentrate on learning the business and your users tasks. You can always re-factor your domain model later. Since you most likely will use a layered or tiered approach, the domain model should have no clue about how to save and retrieve itself from persistence.

Let me emphasize the importance of the Domain Model driving the solution including a Data Model. Consider the Domain Model or Applications in general as being Nature. Consider the Data Model as being human beings. Human beings need nature to survive and live in this world whereas nature does not need human beings to survive. Databases need applications but applications certainly do not need databases. A database without an application is rendered useless whereas an application without a database can still be extremely useful to the user. An application can use different ways to persist the object model or information in general. It does not have to be a database, necessarily.

Technical Challenges

Impedance Mismatch

You can also call Impedance Mismatch a desperate move of old technology trying to survive in an ever-changing world of rapid delivery and instant gratification.

Object-Relational Impedance Mismatch is the difficulty of mapping a Domain Model to a Data Model and vice versa. If you chose to use a relational database as your data storage, you will need to deal with the Impedance Mismatch.

There are several reasons why there is an Impedance Mismatch. The Domain Model is a three dimensional model that includes behavior and data, whereas a Data Model is a flat, two-dimensional model without any behavior. Relational databases do not support inheritance, a crucial feature of object orientation.

On the software development side, impedance mismatch describes the huge differences between the world of objects and the world of databases. Both worlds are also worlds apart where the database world is stuck in a world without much progress and not much innovation. For example, the SQL querying language has not seen any improvements or innovation in the last 20 years.

In the world of objects, it is a thriving world with innovations towards ease of development, productivity, creativity, and automation.

Political / Social Hurdles

The biggest drawback of using Object Persistence is not so much technical but rather political. When you are developing a solution and you are pretty much in control of the entire design and have the freedom to explore different Object Persistence solutions, you might not be faced with the political ramifications. But, if you are in a corporate environment where most corporate environments are hindered to be productive by red tape and personal agendas, introducing an Object Persistence can be somewhat “challenging” to say the least.

Even if you have created the “perfect solution” including an Object Persistence that works, you will still have to potentially face your boss, any cross-team impact including a database team, a mindset of people doing it the “old way” or “the way things have always been done”. You will encounter all kinds of excuses from people even from your boss to convince you, and in many cases order you to do it the old way. By the old way and by the way the database team would want you to do it is that the Database Model is driving the Object Model. Remember, in a domain driven system, it must always be that the Object Model is driving the Data Model (if there is one).

There is hope, though.

Despite the grim outlook when you face the database team, there is a chance that you may be able to introduce an Object Persistence strategy that can work for both teams. Communicating the difficulty of persisting objects and the challenges faced from the software development point of view, for example, can be a pleasant surprise.

Managers / Directors / Executives

Even if some managers try to improve the software development process and the issues around object persistence in the corporate world, it comes down to the executive(s) running the IT department or the people in charge of software development. The ideal scenario is that a CIO fully supports productivity and that he concentrates his efforts in the execution and operation of the IT department and being able deliver to the stakeholders.

You may not be faced with the issues of large corporations if you are working for a smaller company or even if you are self-employed. You will most likely still have to deal with political issues no matter how big or small your company is. 

Summary

Depending on your situation, you may or may not be able to introduce the perfect persistence strategy. But, in todays solutions, you have to deal with persistence one way or another. Architecture is all about compromise. Try to find a compromise that will allows developers to be productive and yet adhere to company guidelines such as a data strategy that a company might already have in place. I welcome very much your feedback and encourage you to so. I future posts I will go into examples and provide possible solutions to this core problem.