Is There a Problem?

 

The fact that you are using objects in your project or product does not mean that the software development process will be any easier. Essentially, you will run into issues that you need to address. In general, some problems will come up before you start designing your solution, some challenges will come up while you are developing your solution, and others issues will need constant attention.

 These issues can be summarized into three categories:

• How Object Persistence has been done in the past
• Technical Challenges
• Political / Social Hurdles

How Object Persistence Has Been Done in the Past

When you ask a developer about a system that he created and then drill-down into the details such as the “data”, many times you will find a “data-centric” thinking in that person even though he is knowledgeable about object-orientation. You will hear that person talk about stored procedures, SQL’s, datasets, even the term entities, etc. Unfortunately, you will discover that the data model is actually driving the entire application. There may or may not be a domain model; but, there will be a database model.

In a domain driven system, the Domain Model must always drive the Data Model.  Never, must the Data Model drive the Domain Model.

The Domain Model dictates when and how the database schema needs to change. If you need to make a design change in your domain model, the database model needs to be updated accordingly so that it can support the domain model. No compromises in the Domain Model must be taking place to support the database model in any way.

The Domain Model should be designed without any regard to object persistence or user interface exposure.

Think of it like this: Design your domain model as if you have unlimited storage space available and all the horsepower in the world to support your object model. Concentrate on solving the domain problems. Concentrate on solving the use cases. Concentrate on learning the business and your users tasks. You can always re-factor your domain model later. Since you most likely will use a layered or tiered approach, the domain model should have no clue about how to save and retrieve itself from persistence.

Let me emphasize the importance of the Domain Model driving the solution including a Data Model. Consider the Domain Model or Applications in general as being Nature. Consider the Data Model as being human beings. Human beings need nature to survive and live in this world whereas nature does not need human beings to survive. Databases need applications but applications certainly do not need databases. A database without an application is rendered useless whereas an application without a database can still be extremely useful to the user. An application can use different ways to persist the object model or information in general. It does not have to be a database, necessarily.

Technical Challenges

Impedance Mismatch

You can also call Impedance Mismatch a desperate move of old technology trying to survive in an ever-changing world of rapid delivery and instant gratification.

Object-Relational Impedance Mismatch is the difficulty of mapping a Domain Model to a Data Model and vice versa. If you chose to use a relational database as your data storage, you will need to deal with the Impedance Mismatch.

There are several reasons why there is an Impedance Mismatch. The Domain Model is a three dimensional model that includes behavior and data, whereas a Data Model is a flat, two-dimensional model without any behavior. Relational databases do not support inheritance, a crucial feature of object orientation.

On the software development side, impedance mismatch describes the huge differences between the world of objects and the world of databases. Both worlds are also worlds apart where the database world is stuck in a world without much progress and not much innovation. For example, the SQL querying language has not seen any improvements or innovation in the last 20 years.

In the world of objects, it is a thriving world with innovations towards ease of development, productivity, creativity, and automation.

Political / Social Hurdles

The biggest drawback of using Object Persistence is not so much technical but rather political. When you are developing a solution and you are pretty much in control of the entire design and have the freedom to explore different Object Persistence solutions, you might not be faced with the political ramifications. But, if you are in a corporate environment where most corporate environments are hindered to be productive by red tape and personal agendas, introducing an Object Persistence can be somewhat “challenging” to say the least.

Even if you have created the “perfect solution” including an Object Persistence that works, you will still have to potentially face your boss, any cross-team impact including a database team, a mindset of people doing it the “old way” or “the way things have always been done”. You will encounter all kinds of excuses from people even from your boss to convince you, and in many cases order you to do it the old way. By the old way and by the way the database team would want you to do it is that the Database Model is driving the Object Model. Remember, in a domain driven system, it must always be that the Object Model is driving the Data Model (if there is one).

There is hope, though.

Despite the grim outlook when you face the database team, there is a chance that you may be able to introduce an Object Persistence strategy that can work for both teams. Communicating the difficulty of persisting objects and the challenges faced from the software development point of view, for example, can be a pleasant surprise.

Managers / Directors / Executives

Even if some managers try to improve the software development process and the issues around object persistence in the corporate world, it comes down to the executive(s) running the IT department or the people in charge of software development. The ideal scenario is that a CIO fully supports productivity and that he concentrates his efforts in the execution and operation of the IT department and being able deliver to the stakeholders.

You may not be faced with the issues of large corporations if you are working for a smaller company or even if you are self-employed. You will most likely still have to deal with political issues no matter how big or small your company is. 

Summary

Depending on your situation, you may or may not be able to introduce the perfect persistence strategy. But, in todays solutions, you have to deal with persistence one way or another. Architecture is all about compromise. Try to find a compromise that will allows developers to be productive and yet adhere to company guidelines such as a data strategy that a company might already have in place. I welcome very much your feedback and encourage you to so. I future posts I will go into examples and provide possible solutions to this core problem.

 

Update on my book

I have increased my efforts to complete my draft for my book “The Abundance of Object Persistence“. The increased efforts are mostly through additional time provided by my current employer RDA Corporation. I will continue small blog posts around the object persistence topic based on my book. I will post these entries at the RDA Architecture Evangelist Team Blog which I am part of as well.

If you have ideas or comments, please provide them. I welcome different point of views as this is usually a hot topic in the IT community.

Funny wrap – Mac vs PC

I was laughing so hard. You have to see and hear this one.

dVP 2009 Award

I’ve been notified that I have been selected again as a db4o Most Valued Professional (dVP) awardee for the year 2009. I’m honored and proud to receive this recognition for the second time in a row. I want to thank db4o and the db4o community. There are only 12 dVP 2009 awardee’s in the USA. Thank you.

Benefits of Object Persistence

When Object Persistence is done right, there can be some great rewards in the areas of Productivity, Maintainability, and Cost Reduction. Some areas directly affect other areas. For example, if one can maintain an application fairly easily, this would result in a higher productivity. There are fine balances between these areas.

 
Productivity

Productivity in software development can be measured in how fast a developer can complete a given work item with the assumption that the quality of the software is not compromised. How much time does it take to get things done without any compromises? Because most projects have a limited amount of time to complete, being productive impacts Cost Reduction to create and maintain the application as well as a better chance for future funding. Being productive should mean that one can get the work done and still deliver high quality software, with a high degree of reliability, and a fair amount of maintainability. Compromises in any one of these areas will result in loss of productivity either in the short-term or long-term of the product life-cycle.

The Object Persistence can be designed in such a way that the system is not locked into a particular database, for example. Because there is no database vendor lock-in, you can gain productivity by simply abstracting storage specific features. For example, if you decide to use an Object Relational Mapping Tool (ORM), you should let the ORM tool worry about creating and maintaining the database schemas for any database it supports based on your Object Model. This can buy you a tremendous amount of productivity because you are delegating this huge maintenance burden to the ORM tool.

• Productivity in software development can be gained through abstraction and automation.

Maintainability

Depending how Object Persistence is done, it usually allows for easier maintainability when requirements change. The fewer changes that need to be made to a system, the easier it is to maintain. Because of abstracting the object persistence, there is a high possibility that a system can be created that is easy to maintain. Because you no longer have to make changes to a database schema or data access layer, you can concentrate on making the changes usually only in one place: the object model and sometimes to the output such as the user interface or a new report

How many times have you read somebody else’s code and trying to figure out what the heck it is trying to do? Hard to read code, or Spaghetti code, is hard to maintain. In general, the fewer lines of code to maintain, the better. Because of the abstraction of the Object Persistence, much fewer lines of code are necessary. A simple method call such as the Save() method is all it’s needed. This makes the code much easier to read and to maintain.

By eliminating the database schema, eliminating a data access layer, and any data or value objects, we are eliminating several points of failures. By reducing point of failures, the system becomes immediately easier to maintain and more reliable.

The easier it is to maintain an application, the less time is required to train somebody else to learn it. Easy to understand code is easy to support no matter who will need to take over the code.

Maintainability is probably the biggest impact of an application throughout its lifetime much more so than the advantages of reusability in Object Orientation.

Cost Reduction

Studies have shown that in the lifetime of an application, 10% of the time is used to create the application, and 90% is used to maintain it until it is retired. If we can reduce the amount of time needed to maintain the application, we can reduce the amount of money spent to keep it alive. We are trying to work smarter using the right tools instead of harder using no tools or unnecessary procedures.

Being more productive has an immediate impact on cost. When a developer can implement a change quicker, the cost of maintaining an application is reduced. The time saved can be used for other duties or other projects. Many times, being more productive can allow for future projects to be approved vs. projects that won’t make the approval because the timelines would not permit to do so otherwise.

Today, labor cost is one of the most expensive liabilities a company can have. On the other hand, the workforce is also the most important asset a company has. There was a time where hardware was the most expensive part in an IT department. Hardware and tools are now extremely inexpensive today. It is mass produced and no longer a major factor in software development.

Purchasing third party component libraries can reduce the time and effort to complete an application. Some of these tools such as ORM tools can seem expensive initially, but; in the long term, you cannot afford not buying them. It is far more costly to write these components or tools on your own than to purchase them.

I’m a Mac User – I’ve done it

I’ve done it. After almost 18 years, I’ve switched from a PC to my new iMac OS X. I still can’t believe it. Over the years, I’ve always monitored Apple and how they went from a proprietary hardware system to the Intel platform with the new OS X.

As a professional, I’m running my .Net development on the iMac using VMWare under a Windows XP window. Works great. The main reason to actually go out and invest into an iMac was iPhone development for the new iPhone 3G using the COCOA Touch framework and XCode.

This is really exciting and I strongly believe that the iPhone from a development point of view has a huge potential. I’ve some personal projects I’m working on for the iPhone at the moment. When the first app is released in the AppStore, I will announce it here. I’ve not felt this excited since the late 80’s when I started writing my first shareware program ACZAR.

What is Object Persistence

In the last few decades, there have been different ways of creating software solutions from procedural top-down development to object oriented and service oriented solutions. They all have their place and were created based on different requirements, budgets, and skill sets. There are always different ways on how a software solution can be created and implemented. One way of implementing a software solution is to use Object Oriented Software Development (OOSD) and its many advantages. If you are on a project that does not necessarily take advantage of OOSD, that’s fine, too. Not all projects may require a full OOSD solution. In addition, you can always refactor parts of a solution to take advantage of OOSD techniques if time and budget permit or if new business requirements are so that it calls for a OOSD solution.

Object Oriented Software Development is a powerful and proven technique to create software solutions that mimic a network of real life processes, tasks, and business rules of a domain. Most importantly, as you design and develop your solution, you start thinking in a domain specific language. You acquire a deeper understanding of the business. Domain Driven Design (DDD) is a technique within OOSD that allows you to get very intimate with the business and requirements at hand. Eric Evans, author of Domain Driven Design, explains in great detail on how this can be accomplished. I highly recommend his book if you are interested in this exciting technique.

The premise of domain-driven design is two-fold:

* For most software projects, the primary focus should be on the domain and domain logic; and

* Complex domain designs should be based on a model.

Domain-driven design is not a technology or a methodology. It is a way of thinking and a set of priorities, aimed at accelerating software projects that have to deal with complicated domains.

To accomplish that goal, teams need an extensive set of design practices, techniques and principles.

The Challenge of Complexity

Of course, many things can put a project off course, bureaucracy, unclear objectives, lack of resources, to name a few, but it is the approach to design that largely determines how complex software can become. When complexity gets out of hand, the software can no longer be understood well enough to be easily changed or extended. By contrast, a good design can make opportunities out of those complex features.

Some of these design factors are technological, and a great deal of effort has gone into the design of networks, databases, and other technical dimension of software. Books have been written about how to solve these problems. Developers have cultivated their skills.

Yet the most significant complexity of many applications is not technical. It is in the domain itself, the activity or business of the user. When this domain complexity is not dealt with in the design, it won’t matter that the infrastructural technology is well-conceived. A successful design must systematically deal with this central aspect of the software.

Source: http://domaindrivendesign.org

Many books have been written about OOSD and I encourage you to dig deeper if you are interested in learning specifics. Besides Eric Evans’ book about Domain Driven Design and Domain Driven Design Quickly, I have also been recommending The Object Primer, by Scott Ambler. I highly recommend his Object Primer and urge you to find some time for it.

What is Object Persistence?

If you are on a project that takes advantage of OOSD techniques, chances are that you also had to worry about storing your domain objects at some point.

When you are dealing with a domain model, business objects working in harmony with other business objects, you will need to deal with Object Persistence sooner rather than later. Object Persistence can be described as a running domain model to survive power failures or power spikes so that no data is lost. Some describe Object Persistence as serializing objects out to the disk and retrieving them later. Serializing an object usually means to store the values of public properties of an object. De-serializing would be just the opposite and retrieve the values and reconstruct the original object back to its original state. Others describe Object Persistence as storing objects into a RDBMS where you will encounter the difficulties of the impedance mismatch between the three dimensional object worlds and flat, two dimensional relational databases equivalent of trying to fit a square into a round hole.

In short, Object Persistence is storing the data, the behavior, and the relationships with other objects for later reuse to output valuable information.

A typical Object Persistence can easily consume 50% – 60% of a project’s resources and time if not handled correctly. This time is not just the initial time when a system is being developed. More importantly, the future maintenance of a system is the most expensive part of a system during its lifetime until retirement. If Object Persistence takes up such a big part, shouldn’t Object Persistence deserve a lot more attention?

There are several ways on how Object Persistence can be done. I will go into some details in future posts. Until then, think about the time and productivity that can be gained through effective Object Persistence and not just during prototyping but full blown production systems. How much time do you usually spend dealing with creating a database schema, creating some layer to store “stuff” into tables, and how much time you spend updating all these parts until the system goes into production. Once in production, how much time do you spend making changes as requirements change? How much plumbing work do you do vs. actually solving the domain problems?

In future blogs, I will post about Object Persistence and why this is so important in software development projects that do utilize OOSD today. I will take ideas from my upcoming book The Abundance of Object Persistence to highlight the importance of this crucial part in system architecture not only from a technical perspective but also from a business point of view. Please feel free to provide any comments or suggestions. I’m looking forward to your feedback.

New Demo

I have found that quite a few people are using my presentation demo as a guide to start new projects. This was not my intention but rather a very simple example on how you could separate the layers and use db4o.

I realize that it is easier to learn by example especially on topics such as design patterns, persistence, layering a system, etc. So, I decided to start working on a new demo. This new demo will show you how you can effectively layer your system, how you can separate your persistence, how you unit test, implement several design patterns etc.

It will definitely help you to get a major head start when starting a new project. Please let me know if you have any ideas or special cases you would like to consider for the demo.

Object Persistence Presentation in Columbia, SC

I will present at the Columbia Enterprise Developers Guild in Columbia, SC on Wednesday, September, 12th, 2007 at 6pm at the Midlands Tech NE Campus Auditorium. I will be presenting db4o as a means to easily persist your objects in .Net. I hope to see you there and enjoy the sugar bus.

dVP 2008 Award

I’ve just been notified that I have been selected as a db4o Most Valued Professional (dVP) awardee for the year 2008. I’m honored and proud to receive this recognition. I want to thank db4o and the db4o community and I’m looking forward to an even stronger db4o, and Object Persistence in general, acceptance in small and large companies worldwide.