Patterns of Enterprise Application Architecture
My latest work, and I think the hardest one to write. It all started after Dave Rice and I gave some talks on J2EE architecture and mulled over how the concepts we had learned in C++, Forte, CORBA, and Smalltalk had been crucial to us developing good designs in Java. With this book I wanted to set many of these patterns down to help developers whatever their platform. As it's turned out these have been very valuable as we have started to use .NET in 2002.
The book is in two parts. The first part is a short (100 page) tutorial on enterprise application architecture. The bulk of the book is the reference to forty or so patterns. All of these patterns are ones that I've seen in the field, usually on many different programming platforms.
Each pattern describes details of how it works and when to use it, together with code examples in Java, C# or both. (I picked these languages because most software developers can read them, not because of any particular coupling or opinions about the language).
The main topic areas are: how to layer an enterprise application, how to organize domain logic, how to tie that logic to a relational database, how to design a web based presentation, some important principles in distributed design, and handling of what we call "offline concurrency" - concurrency that spans transactions.
I've been fortunate enough to have some excellent contributors to this book, most notably Dave Rice - who wrote a good tenth of it. Also joining in are Matt Foemmel, Edward Hieatt, Robert Mee, and Randy Stafford
The book has won two awards: a productivity award from Software Development magazine and a best java book award from JavaWorld.com.
Translations: Portuguese, Polish, German, Russian, Korean, Japanese.
Further Reading
- Catalog of Patterns
- Brief summaries of the patterns in the book
- Distribution Strategies Chapter
- SD Magazine adapted chapter 7 as an article for the magazine (Registration required with CMP)
Refactoring: Improving the Design of Existing Code
Refactoring is a controlled technique for improving the design of an existing code base. Its essence is applying a series of small behavior-preserving transformations, each of which "too small to be worth doing". However the cumulative effect of each of these transformations is quite significant. By doing them in small steps you reduce the risk of introducing errors. You also avoid having the system broken while you are carrying out the restructuring - which allows you to gradually refactor a system over an extended period of time.
My book describes the process of refactoring and spends most of its time explaining how to do the various refactorings - the behavior preserving transformations. The book opens with a simple example that describes the whole process. There are then some introductory chapters that discuss broader issues around refactoring, the "code smells" that suggest refactoring, and the role of testing.
The bulk of the book is around seventy refactorings described in detail: the motivation for doing them, mechanics of how to do them safely and a simple example.
The books finishes with some guest chapters from those whose early work have developed the ideas of refactoring.
Translations: Japanese, German, Polish, Serbian, Portuguese, Chinese (Simplified and Traditional), Russian, Czech, and Korean
Further Reading
- refactoring.com
- My original hope was to make this a portal for on-going work on refactoring. I've learned that my ambitions never quite match up to the time I have available. However here you will find links to further resources on refactoring, including the mailing list and tools as they become available. I've also got a catalog that includes summaries of all the refactorings in the book, together with some new ones.
UML Distilled: A Brief Guide to the Standard Object Modeling Language
The UML appeared in 1997 to eliminate the bedlam that had overtaken graphical modeling languages in the object-oriented world. Before the UML there were a host of such languages, differing in all sorts of annoying ways. Now the UML is pretty much the only game in town, which makes life much simpler for many people involved in OO software development
This book was the first book on the UML, and I wrote it as a short overview of the language. My aim was to target that fraction of the UML that was the most useful and explain it briefly. My proudest thing about this book is that it's under half an inch thick - which was very unusual for OO modelling books at that time. I was also pleased when it won a productivity award from Software Development magazine.
The book has chapters for all the major diagram types in the UML, together with an intro and some general guidance on software process and how it fits in with the UML. I also put in a lot of tips on how to use the UML well and introduce a number of principles and techniques for good OO development.
In September 2003, the 3rd edition of this book came out. This edition was mainly put together to bring the book into line with the latest UML 2 standard. I've further taken the opportunity to completely rewrite most of the book, clarifying explanation from my experiences and from review comments from others. I've added chapters for the new diagram types that were added in UML 2. However by slimming down the discussion on process, and removing the final code example chapter, I've actually managed to reduce the page count of the book. (It looks thicker because AW printed it on thicker paper.)
Translations: Third Edition: Korean, Chinese (Simplified and Traditional), Indonesian, Japanese, Russian, Portuguese, Bulgarian, Italian, French, Serbian, and German. Second Edition: Japanese, German, Dutch, Polish, Russian, Portugese, Korean, Chinese. First Edition: French, Spanish
Further Reading
- Supplementory Articles
- Some extra material about the UML on my site, including the summary of the changes in UML 1.x from the book.
Planning Extreme Programming
In 1996 I was lucky enough to work on the C3 project for a year - this was the project that gave birth to Extreme Programming. I learned many things by taking part in that project, but one thing that particularly impressed me was the approach to planning. Not just was the project run on firmly iterative lines, it included a planning process that was simple, agile and remarkably effective.
After Kent's seminal "white book" came out, we talked about what kinds of books should follow on. Somehow Kent and I talked each other into collaborating on a book that focused on the planning side of Extreme Programming. Although it was based on the C3 experience, it also added lessons both we, and other early XPers, had learned from the first wave of XP projects.
Now there are quite a few XP books out there, and they are actually a very good bunch. But one valid complaint is that they are all somewhat similar. As a result, if you already have several XP books I'm not sure that you'll much that's new in this one. I think the primary value of this book lies in its sole focus on planning. If you're in a management or customer role on an XP project, then the technical practices don't matter that much to you. In which case a book that concentrates solely on the planning side is a good one to pick.
I also think that this is a good book even if you aren't into XP itself, but you do want to plan an iterative and agile project. One of the features of agility is that you recognize that software development is rarely a predictable process. But often that results in a different problem - where people don't really plan at all, or do plans that have little relationship with reality. The nice thing about XP's planning scheme is that it gives you control in a changing environment. So while you lose predictivity you don't lose control - and control is essential to successfully running an iterative project.
Translations: Japanese, German, Chinese (Simplified and Traditional), Russian
Further Reading
- Links
- My links page links to various further resources on XP and agile methods
- Articles
- I've written several articles about agile software development
- The New Methodology
- An outline article that explains what agile methods are all about
- Manifesto for Agile Software Development
- I was one of the authors of this surprisingly influential polemic
Analysis Patterns: Reusable Object Models
My first book, and yes I admit it's showing its age. But it's still one of very few books that have tried to tackle this kind of material. It's about trying to find patterns in domain models of businesses, and is based on much of the domain modeling work that I did (and still love doing) in the late 80's and early 90's.
The patterns come from various domains, including health care, financial trading, and accounting. Each of the patterns is described both textually and in a simple pre-UML notation (this book was written before the UML had stabilized into a usable form).
There's much I could do to improve the presentation of the patterns in this book with what I've learned since. The core patterns themselves, have stood up surprisingly well to the test of time. There's very little here in the core content that I would change, but there's much I could add and I think I could now make them a lot easier to understand.
Translations: German, Japanese, Chinese (Simplified)
Further Reading
- Articles
- At one point I did embark on reworking some of the patterns and adding new ones. That effort has stalled for a while, I hope to pick it up someday. Until then, here are some of the more up to date ideas and presentations of the patterns.
My Signature Series
In 2002 Addison-Wesley launched their signature series. The idea is that their leading authors would host a small series of books that they felt were particularly special. Kent Beck and I were selected to be the first "signers". In this series I'm getting involved in, and recommending, books that I wish I could have written.
Beyond Software Architecture: Creating and Sustaining Winning Solutions
by Luke Hohmann
Architecture has become a very slippery word in the software business. It's hard to come up with any solid definition of what it means. I see it as a fundamentally subjective term - when people describe their software architecture they select the important parts of their systems, how these parts fit together, and the key decisions they made in designing their systems. Architecture is also seen as a technical issue, with the implication that the key decisions that need to be made are technical decisions.
In talking with Luke over the last few years I've really enjoyed the fact that he talks about the kinds of things that are often sadly omitted from most architectural discussions - yet are every bit as important. Such things as the marketing view of a system, licensing terms, branding, deployment, billing. All of these issues have important technical and business implications. Senior technical people need to think about this stuff, or otherwise a technically capable system could fail to be good business decision.
Many of these issues matter most to people who sell software to other entities. But even if you're an architect of an in-house IS shop these issues are just as likely to trip you up. Licensing agreements with your vendors can make a big difference to the costs of the software you deploy, billing may become important if your business decides it wants to introduce a charge-back scheme, branding helps affect your visibility to the business side of your company.
Luke writes from the perspective of someone who has dealt with both the technical and business sides of software development. It's a duality I find appealing because it's led him to ponder issues that often don't get talked about. He shows that it's often the things you don't think to worry about that hurt you the most and in the process provides the advice you need to deal with them. As a result this book is a much needed complement to the technical aspects of software design.
Further Reading
- Luke's Home Page
- Here you can find more information about the book, as well as other papers that Luke has written.
Enterprise Integration Patterns : Designing, Building, and Deploying Messaging Solutions
While I was working on P of EAA, I was lucky to get some in-depth review from Kyle Brown and Rachel Reinitz at some informal workshops at Kyle's office in Raleigh-Durham. During these sessions, we realized that a big gap in my work was asynchronous messaging systems.
There are many gaps in my book, and I never intended it to be a complete collection of patterns for enterprise development. But the gap on asynchronous messaging is particularly important because we believe that asynchronous messaging will play an increasingly important role in enterprise software development, particularly in integration. Integration is important because applications cannot live isolated from each other. We need techniques that allow us to take applications that were never designed to interoperate and break down the stovepipes so we can gain a greater benefit than the individual applications can offer us.
Various technologies have been around that promise to solve the integration puzzle. We all concluded that messaging is the technology that carries the greatest promise. The challenge we faced was to convey how to do messaging effectively. The biggest challenge in this is that messages are by their nature asynchronous, and there are significant differences in the design approaches that you use in an asynchronous world.
I didn't have space, energy, or frankly the knowledge to cover this topic properly in P of EAA . But we came up with a better solution to this gap: find someone else who could. We hunted down Gregor and Bobby, and they took up the challenge. This book is the result.
I'm delighted with the job that they have done. If you've already worked with messaging systems, this book will systematize much of the knowledge that you and others have already learned the hard way. If you are about to work with messaging systems, this book will provide a foundation that will be invaluable no matter which messaging technology you have to work with.
Further Reading
- Enterprise Integration Patterns Web Site
- Gregor maintains a site with a lot more information on enterprise integration with messaging.
Refactoring to Patterns
For several years now, I've been involved with advocating agile methods in general, and Extreme Programming in particular. When I do people often question how this fits in with my long running interest in design patterns. Indeed I've heard people claim that by encouraging refactoring and evolutionary design, I'm recanting what I've previously written about analysis and design patterns.
Yet all it takes is a quick look at people to realize that this view is flawed. Look at the leading members of the patterns community and at the leading members of the agile and XP communities and you see a huge intersection. The truth is that patterns and evolutionary design have had a close relationship since their very beginnings.
Josh Kerievsky has been at the heart of this overlap. I first met him when he organized the successful patterns study groups in New York City. These groups did a collaborative study of growing literature on design patterns. I quickly learned that Josh's understanding of design patterns was second to none, and I gained a lot of insight into those patterns by listening to him. Josh adopted refactoring early, and was an extremely helpful reviewer on my book. As such it was no surprise to me that he also was a pioneer of Extreme Programming. His paper on patterns and Extreme Programming at the first XP conference is one of my favorites.
So if anyone is perfectly suited to write about the interplay of patterns and refactoring, Josh is. It's territory I explored a little bit in Refactoring, but I didn't take it too far because I wanted to concentrate on the basic refactorings. This book greatly expands that area discussing in good detail how to evolve most of the popular Gang of Four patterns, showing that they need not be designed in up front, but evolved to as a system grows.
As well as the specific knowledge about these refactorings that you can gain from studying them, this book also tells you more about patterns and refactoring in general. Many people have said they find a refactoring approach to be a better way of learning about patterns, because you see in gradual stages the interplay of problem and solution. These refactorings also reinforce the critical fact that refactoring is all about making large changes in tiny steps.
So I'm delighted to be able to present this book to you. I've spent a long time cajoling Josh to write a book, and then working with him on this one. I'm delighted with the result and I think you will be too.
This book won a Software Development Magazine Productivity award in 2005.
Further Reading
Refactoring Databases : Evolutionary Database Design
A decade ago 'refactoring' was a word only known to a few people, mostly in the Smalltalk community. It's been wonderful to watch more and more people learn how to use refactoring to modify working code in a disciplined and effective manner. As a result many people now see code refactoring as an essential part of software development.
I live in the world of enterprise applications, and a big part of enterprise application development is working with databases. In my original book on refactoring I picked out databases as a major problem area in refactoring since refactoring databases introduces a new set of problems. These problems are exacerbated by the sad division that's developed in the enterprise software world where database professionals and software developers are separated by a wall of mutual incomprehension and contempt.
One of the things I like about Scott and Pramod is that, in different ways, they have both worked hard to try and cross this division. Scott's writings on databases have been a consistent attempt to bridge the gap, his work on object-relational mapping has been a great influence on my own writings on enterprise application architecture. Pramod may be less known, but his impact has been just as great on me. When he started work on a project with me at ThoughtWorks we were told that refactoring of databases was impossible. Pramod rejected that notion, taking some sketchy ideas and turning them into a disciplined program that kept the database schema in constant, but controlled motion. This freed up the application developers to use evolutionary design in the code too. Pramod has since taken these techniques to many of our clients, spreading them around our ThoughtWorks colleagues and, at least for us, forever banishing databases from the list of roadblocks to continual design.
This book assembles the lessons of two people who have lived in the no-mans land between applications and data and presents a guide on how to use refactoring techniques for databases. If you're familiar with refactoring you'll notice that the major change is that not just do you have to change program and data structures, you also have to manage continual migration of the data itself. This book tells you how to do that, backed by the project experience (and scars) that these two have accumulated.
Much though I'm delighted by the appearence of this book, I also hope it's only a first step. After my refactoring book appeared I was delighted to find sophisticated tools appear with automated many refactoring tasks. I hope the same thing happens with databases, and we begin to see vendors offer tools that make continual migrations of schema and data easier for everyone. Before that happens this book will help you build your own processes and tools to help; afterwards this book will have lasting value as a foundation for using such tools sucessfully.
Further Reading
- Database Refactoring Website
- Website for the book with summaries of the refactorings.
Continuous Integration: Improving Software Quality and Reducing Risk
In my early days in the software industry, one of the most awkward and tense moments of a software project was integration. Modules that worked individually were put together and the whole usually failed in ways that were infuriatingly difficult to find. Yet in the last few years, integration has largely vanished as a source of pain for projects, diminishing to a non-event.
The essence of this transformation is the practice of integrating more frequently. At one point a daily build was considered to be an ambitious target. Most projects I talk to now integrate many times a day. Oddly enough it seems that when you run into a painful activity, a good tip is to do it more often.
One of the interesting things about Continuous Integration is how often people are surprised by the impact that it has. We often find people dismiss it as a marginal benefit, yet it can bring an entirely different feel to a project. There is a much greater sense of visibility, because problems are detected faster. Since there is less time between introducing a fault and discovering you have it, the fault easier to find because you can easily look at what's changed to help you find the source. Coupled with a determined testing program, this can lead to a drastic reduction in bugs. Developers as a result spend less time debugging and more time adding features, confident they are building on a solid foundation.
Of course it isn't enough simply to say that you should integrate more frequently. Behind that simple catch phrase is a bunch of principles and practices that can make continuous integration a reality. You can find much of this advice scattered in books and on the Internet (and I'm proud to have helped add to this content myself), but you have to do the digging yourself.
So I'm glad to see that Paul has gathered this information together into a cohesive book, a handbook for those that want to put together this best practice. Like any simple practice, there's lots of devil in the details. Over the last few years we've learned a lot about those details and how to deal with them. This book collects together these lessons to provide as solid a foundation for Continuous Integration as Continuous Integration does for software development.
xUnit Test Patterns: Refactoring Test Code
If you go to junit.org, you'll see a quote from me: "never in the field of software development have so many owed so much to so few lines of code". JUnit has been criticized as a minor thing, something any reasonable programmer could produce in a weekend. This is true, but utterly misses the point. The reason JUnit is important, and deserves the Churchillian knock-off, is that the presence of this tiny tool has been essential to a fundamental shift for many programmers. A shift where testing has moved to a front and central part of programming. People have advocated it before, but JUnit made it happen more than anything else.
It's more than just JUnit, of course. Ports of JUnit have been written for lots of programming languages. This loose family of tools, often referred to as xUnit tools, have spread their way far beyond the java roots. (And of course the roots weren't really in Java, as Kent Beck wrote this code for Smalltalk years before.)
XUnit tools, and more importantly the philosophy, offer up a huge opportunity to programming teams. An opportunity to write powerful regression test suites that enable teams to make drastic changes to a code-base with far less risk. Opportunities to re-think the design process with Test Driven Development.
But with these opportunities come new problems and new techniques. Like any tool, the xUnit family can be used well or badly. Thoughtful people have figured out various ways to use xUnit, to organize the tests and data effectively. Like the early days of objects, much of the knowledge to really use the tools is hidden in the heads of its skilled users. Without this hidden knowledge you can't really get the full benefits.
It was nearly twenty years ago when people in the object-oriented community realized this problem for objects and began to formulate an answer. This answer was to describe their hidden knowledge in the form of patterns. Gerard Meszaros was one of the pioneers in doing this. When I first started exploring patterns, Gerard was one of the leaders that I learned from. Like many in the patterns world, Gerard also was an early adopter of Extreme Programming, and thus worked with xUnit tools from the earliest days. So it's entirely logical that he should have taken on the task of capturing that expert knowledge in the form of patterns.
I've been excited by this project since I first heard about it. (I had to launch a commando raid to steal this book from Bob Martin because I wanted it to grace my series instead.) Like any good patterns book it provides knowledge to new people in the field, and just as important, provides vocabulary and the foundations for experienced practitioners to pass their knowledge onto their colleagues. For many people, the famous Gang of Four booked unlocked the hidden gems of object-oriented design, this book does the same for xUnit.
--
Happy day, happy life!
No comments:
Post a Comment