Monday, September 13, 2010

Rereading Mythical Manmonth - Flowcharts and UML

This is a second entry about Rereading Mythical Manmonth. You can read the first one here.

When I started programming, I never used to create 'detailed flow charts' while many people (usually who are not actively coding) urged it. I did create some small flowchart to get overall picture in my mind but never detailed flowcharts. Later same thing happened with UML diagrams, I always felt little guilty about it. Till I read, Fred Brooks observations about Flow charts.

These are the Fred Brooks observations about Flow Charts in section named 'Flow Chart Curse' (in essay 15, The Other Face).  
The flow chart is a most thoroughly oversold piece of program documentation. Many programs don't need flow charts at all; few programs need more than a one-page flow chart. 
The detailed blow-by-blow flow chart, however, is an obsolete nuisance, suitable only for initiating beginners into algorithmic thinking.
In fact, flow charting is more preached than practiced. I have never seen an experienced programmer who routinely made detailed flow charts before beginning to write programs. Where organization standards require flow charts, these are almost invariably done after the fact. Many shops proudly use machine programs to generate this "indispensable design tool" from the completed code. I think this universal experience is not an embarrassing and deplorable departure from good practice, to be acknowledged only with a nervous laugh. Instead it is the application of good judgment, and it teaches us something about the utility of flow charts.
Try replacing the 'Flow chart' in the above discussion with 'UML Diagrams' and the whole discussion now suddenly seems very recent discussion. In my experience 'UML Diagrams' are more preached that practiced and most of the time these diagrams are created 'automatically' from the code at the end of development cycle.

Obviously with UML also, I create a few class diagrams and sequence diagrams to clear top level picture. But I don't create very detailed UML diagrams of every single class and function. In my experience, when a team tries to create very granular UML diagrams :
  • it confuses every one in the team, 
  • it takes too much efforts 
  • and hence not really worth it.
However, many times Customers and internal SEPG groups ask for UML diagrams (sometimes because of mostly misguided notion that a UML documented design means 'good design', sometimes because of everybody is doing it, sometimes because of all the UML hype).

Still I could never clearly explain why UML is not a 'silver bullet' in documenting software design. That is till I reread the 'No Silver Bullet' and the section 'Essential difficulties->Invisibility'.
Software is invisible and unvisualizable. Geometric abstractions are powerful tools. The floor plan of a building helps both architect and client evaluate spaces, traffic flows, views. Contradictions become obvious, omissions can be caught. Scale drawings of mechanical parts and stick-figure models of molecules, although abstractions, serve the same purpose. A geometric reality is captured in a geometric abstraction.

The reality of software is not inherently embedded in space. Hence it has no ready geometric representation in the way that land has maps, silicon chips have diagrams, computers have connectivity schematics. As soon as we attempt to diagram software structure, we find it to constitute not one, but several, general directed graphs, superimposed one upon another. The several graphs may represent the flow of control, the flow of data, patterns of dependency, time sequence, name-space relationships. These are usually not even planar, much less hierarchical. Indeed, one of the ways of establishing conceptual control over such structure is to enforce link cutting until one or more of the graphs becomes hierarchical.

In spite of progress in restricting and simplifying the structures of software, they remain inherently unvisualizable, thus depriving the mind of some of its most powerful conceptual tools. This lack not only impedes the process of design within one mind, it severely hinders communication among minds.
Since software is inherently 'unvisualizable' (as explained above), software developer will always feel that any graphical modeling technique (Flow charts, UML Diagrams, Swimlanes, data flow diagrams, etc etc) is insufficient to imagine and communicate the software design.  

So in conclusion:
  • Software design is hard.
  • Documenting and communicating software design is even harder.
  • Flowcharts, UML diagrams or any other diagrams are not a silver bullet of 'design documentation'.
  • Most likely, there will never be a 'silver bullet' in design documentation.
UPDATE (Oct 30, 2010):
I fully agree that UML diagrams are useful communication tool to document/discuss/understand the overall design. However, it utterly fails as  'documentation' tool for documenting every class and function. Documenting every single class with UML results in information overload. So I agree with observations of 
Stephan.Schmidt and others.

In one of the project that I worked, we had about 10 different modules. Each module had some 2/3 key classes. We created module dependency diagram. We documented the class hierarchy (inheritance and aggregation relationships) of these classes in a single class diagram. We also documented names of various design patterns used in these classes. Then we documented key functions of these modules with 2/3 sequence diagrams. This documentation served us well. New team members can quickly get the hang of the system and start contributing.  
At the end of the project, our customer insisted on UML diagrams for every single class and function. Hence we reverse engineered whole system and created UML documentation of every single class. It was completely incomprehensible and hence completely useless.


    Donal Lafferty said...

    Despite UML's failures, visual techniques are promising solutions to what Brooks termed the 'accidental' aspects of programing. In his book on human memory, Higbee explains Visual-based memory recollection is an order of magnitude faster than text-based. As a result, I would expect that understanding program with visual aids would reduce the learning time by an order of magnitude. Anecdotal evidence supports this view, as using Microsoft's Visual Studio SQL edit allow me to become comfortable with SQL rather quicker than my earlier efforts using text-only aids. Furthermore, in its discussion of 'C', The Mythical Man Month suggests the per instruction performance of programmers is largely fixed. Thus, higher level languages (i.e. 'C' if memory serves me correctly) improved performance by roughly the ratio of high-level instructions to low level instructions. Taken together, these points suggest that for efficencies sake programmers should aim to implement their programs with more visual tools, albeit perhaps not UML directly.

    Secondly, it seems that UMLs use is to better explain software than to actually implement it. In fairness, advance preparation of UML documents give software developers an idea of what the end result might look like. Unfortunately, you've pointed out that by project's end, the UML is merely generally a rough guide to actual software. Furthermore, Brooks aludes to the difficulty that a diagram will only capture one view point of the software, whereas software can be diagrammed from different points of view depending on the cross cutting concern we wish to diagram. However, a general diagram does serve well when trying to explain the architecture or behaviour of software in general terms. Thus, we see power point presentations developed around images complemented with just source code declarations and examples. UML could be seen as a compremise in that it abstracts out a particular programing language, yet offers the ability to capture complexity of architectural elements of interactions. To avoid their construction from becoming burdensome, these images might be a bare and in need of additional explaination, but these limits do not prevent their use in meetings or engineering discussions.

    thatotherdewd said...

    I've found UML useful when getting grips on legacy code that grew organically. Having views from different levels of granularity makes it much easier to asses the impact of changes to realize new stories.

    Stephan.Schmidt said...

    I find UML a excellent tool for sketching solutions and discuss system parts. If utterly fails when one tries to "document" the whole system.


    Unknown said...

    I haven't written detailed UML diagrams for anything since my Software Engineering classes in graduate school. When I did my capstone project, we formed teams of 4 people and had to start to finish come up with and implement a cash register interface, inventory storage system, and sales/inventory management/reporting interfaces. When a team of bumbling programmers has to tackle a large scale project and come up with something that some future team of bumbling programmers has to maintain, I think UML serves its purpose very well.

    You're exactly right that programs are very hard to visualize: the purpose of UML is to help/coerce programmers to cram their program into a metaphor that human beings can read and understand. It is a design style as much as it is a design representation, and I think its a good style. For my personal projects, especially the small ones, a sort of mystic sub-conscious grasp of the entire project suits me just fine, but if someone else were to come along with little knowledge of the project and begin to work on it, they may or may not be totally fucked, depending on how 'representable' the structure of the program is to an entity outside of my own brain. That is the purpose of UML.

    Evan said...

    Flowcharts and UML diagrams are best for computer programming. I have used several flowchart making tools but found craetely to be most creative tool among them all.