Software That Works - A Review
On this page I would like to collect some key points from the book "Software That Works" by Michael Ward, 1990, Academic Press, Inc. San Diego, California 92101, ISBN 0-12-735040-3
I came across this book literally by chance in the library of the University of Rostock. Even though it is already quite old, I think also for more modern software development very useful hints can be derived from it. What I liked especially was that software projects of all sizes are considered - from a one person hack to multimillion dollar governmental projects. The strategies discussed are not especially bound to a certain programming language. On the contrary more general practices of design, organisation and software work are treated.
The author choses a very pragmatical approach: everything that is useful to reach the goal of software that works, is taken, counterproductive things are discarded. As the chapters are quite elaborate and keep repeating and cross linking several topics, I would like to extract kind of the key statements from the chapters (some almost literally), which might be more useful for everyday work.
Please enjoy the book as well, it makes quite a nice reading. We try together with DESY Education Department to get a copy of the book for the "blue point shelf" in the library, as it is out of print for some time already.
Software That Works - A Review
- Chapter 1: Getting Started
- Chapter 2: Issues of Design
- Chapter 3: Structured Methods
- Chapter 4: The Iterative Cycle
- Chapter 5: Prototypes
- Chapter 6: Analysis
- Chapter 7: Requirements
- Chapter 8: Decomposition
- Chapter 9: Dataflow
- Chapter 10: Data Dictionary
- Chapter 11: The State Machine
- Chapter 12: Modularity
- Chapter 13: Data Hiding
- Chapter 14: Interfaces
- Chapter 15: The Module Tree
- Chapter 16: Flow Charts
- Chapter 17: Conventional Wisdom
- Chapter 18: Program Design Language (PDL, Pseudocode)
- Chapter 19: Error Checking
- Chapter 20: Writing Code
- Chapter 21: Testing
- Chapter 22: Designing with Large Projects and Small
- Chapter Last: The Sound of One Hand Clapping
- Appendix A: Small Design Example
- Appendix B: Initial Design of a Large System
Chapter 1: Getting Started
- Use methods as they help you to create working software!
- Don't try to follow methodologies religiously, there is no price for the best followers, only the goal of working software!
Every project (size doesn't matter) has restrictions that might lead to the situation that the result is not worth the effort any more, which will lead to the project being dropped. The way out is to do proper software design (see Chapter 2).
Don't put too much effort in the design up front or in any other phase of software development! Instead, one has to return to the drawing board, again and again. It is almost impossible to come up with a perfect design which can be created, implemented and released - it's an iterative process and that should be regarded when organizing the work (see Chapter 4).
- Time and Energy are the two factors of influence. Energy level can be kept high by changing between phases of work (design, coding) and fun. In each of those one can recreate from the others, keep the fun factor high, stay energetic and motivated.
- Energy is too precious to waste. Don't try to get it right the first time! Don't try to get it right this time!
- Don't waste energy on doing things manually which the PC will do for you anyway (e.g. finding syntax errors) or can help you to do much more efficiently (e.g. finding logical errors).
- Don't work to much, have a little fun, stay flexible, try different things!
Chapter 2: Issues of Design
- Design is communication
- It is the most reliable way to software that works. You think you can do without design? It's a trap! Do it!
- Start before you write the program, return to the design stage while you write the program.
- It is the way from the idea in one's head to paper to implementation, by describing, communicating it. Thus the idea matures and may become practical.
- Design documents (flow charts, documentation, sketches, ...) are snapshots of the design at a certain time, so they may differ! Design can be frozen to get everything up to date again.
- It is not necessary to do design "correctly" according to some standards. It can help.
Example: Many methodologies recommend or insist not to use "goto". This originated from "goto" being misused and code becoming more and more incomprehensible. So following the recommendations will help to get understandable code. On the other hand "goto" sometimes is the most elegant way (e.g. breaking out of several nested loops), so why not use it? As long as the purpose of its use is communicated, everything is fine.
- It is important to do design such, that it communicates the idea to you and your colleagues.
Chapter 3: Structured Methods
- Structured programming uses three key constructs:
- Sequences of commands
- Branching (if...then...else)
- Loops with condition check at the beginning or end
- Jumping commands (goto, exit, break, return, try...catch) can destroy the structure created by the above methods, so they must be treated with special care. In principle they can be avoided completely. On the other hand they can be useful in special cases (breaking out of loops and other nested constructs)
- Another key component is modularization (putting parts of the program into functions, procedures, subroutines with defined interfaces). It breaks a big complex problem into smaller problems that can be easier understood. Thus the code reflects the design ideas more clearly and simplifies documentation.
- Keep the scope of variables as local as possible. By doing so chances are lower to re- or abuse variables in several places while still in use, which is a common source of faulty behaviour.
- Global variables can help to avoid cumbersome variable passing chains within the module tree.
- If global variables are combined with jumping commands, data integrity can be violated. By these commands a part of the code is practically "bypassed". In this way checking procedures could be skipped, variables might have no value that is expected elsewhere in the program.
- Top down design appeared at the same time as structured programming. It means designing from requirements to functions to modules to code. Though very useful in combination, it requires a thorough understanding of the problem and the environment (operating system, programming tools, ...) in order to result in working code.
- If such understanding is only generated while writing code, or the program is based on examples which are extended step by step, a bottom up design approach in combination with structured methods is more promising.
- Last but not least both ways can be combined to reach the goal of working software.
- Structured programs have the drawback of needing more CPU time and memory (administrative overhead); for extreme performance requirements this can be critical!
- However, often this overhead can be tolerated or is even negligible and the full benefit of structured methods can be obtained.
Chapter 4: The Iterative Cycle
- This chapter is about overall organization of work
- Classical approach goes in three steps: from Design to Coding to Debugging. This may look convenient, but it does only for managers and does not work in reality. It implies that the design goes right the first time, which clearly disagrees with the statements of previous chapters.
- Errors result either from Coding or Design. The first ones are found by the compiler, the latter ones can only be cured by adapting the design, so there is no point in having a dedicated "debugging".
- In order to implement a permanent change between Design and Coding an iterative cycle is introduced: design followed by coding and testing. From the last two steps one can and shall go back to design as often as necessary.
- These iterative cycles are maintained during all project phases: Development, Alpha Phase (Testing by friends), Beta Phase (Testing by hostile strangers), Product Release (to the even more hostile and unforgiving public). Transitions between Project phases are not clearly defined so be carefull what you offer to the next phase. Many individuals and companies have survived beta testing in public, but many have not, either.
The point of changing between steps in the iterative cycle is usually when you feel like it. Coding is relaxation from design and vice verser (see Chapter 1).
- In larger groups people will feel like the one or other step at different times, thus a permanent design dialog can be kept going. Likewise regular meetings (formal or informal) can be scheduled for this (coffee breaks are great for this as people from other disciplines can easily join in)
- Even though the iterative approach is known to work, management is often not willing to allow for it, as it takes control from them. People who know whats going on in development should make the decisions, of course. Often this is not the management.
- There are managers who can manage software projects and those who can't. Neither wants to be educated on the matter.
- If management disagrees with the iterative cycle, you can try to do it anyway within the given structure, or change your job. Otherwise you will most likely produce non working software and be made liable for it.
- The lessons of this chapter are an important part of proper design method. It's worth fighting for.
Chapter 5: Prototypes
- Prototyping is a kind of bottom up development from small, well understood parts to a more complex system. It is very useful to get familiar with a problem or environment. Besides it is its only purpose.
- If the entire team is intimately familiar with both, strict top down design can produce excellent programs. As humans are involved it should not be thought of as a linear sequence from requirement to functions to modules to code, but again iterations between these steps should be employed.
- One should strive for a "top downish" overall approach. This can be nicely complemented by bottom up prototyping of heavily used functionality, to get to know the system.
- The first attempted prototypes will be throw away code. It will be full of bugs, not really understood, yet, and thus is not suitable to build further programming on it. Likewise compatibility is not an issue at this stage. So make sure you throw it away afterwards. If the produced prototypes don't work understand why, thus you will learn. Have fun while learning, e.g. by writing a little game, play around with the system.
- After having played around long enough to get familiar with the system, make attempts for more focussed prototypes. They will already go roughly into the direction of the planned project. Try to thoroughly understand these key components of the later system. Make sure they work under the controlled conditions of the prototype. Stress-test them in a prototype, think of unexpected conditions that could appear (invalid data) and how to cope with it.
- Use prototypes for a proof of concept, but don't make them working horses for presentable results.
- Try to break the prototype, let others try to break it. If it has become good enough throw it away. Do not try to build software directly on such prototypes. They were only intended for learning!
- It is the essence of prototypes that they be abandoned.
- The only outcome of prototyping is the knowledge gained by working on and with them. Based on this knowledge a proper top down design can be started. Key components of the system can be rewritten in a much more adequate way, now problem and environment are much better understood. When the top down approach reaches the coding level it will fit much better with the programming environment.
- So however you continue, make a fresh start and throw away the prototypes.
Chapter 6: Analysis
- Analysis was a separate job in the early days of software development (Analysis of the Problem by Analyst, software-design by Designer, Coding by Programmer)
- Nowadays these tasks are often combined
Analysis as a first step in design means the process of finding a description the given problem by breaking it into smaller, possibly well known problems (Decomposition). There can be several different results from this process.
- After breaking down the problem, it is helpful to categorize the parts. General categories, like input, process, output or initialize, decide, process, terminate, can be used.
- If you are more experienced, you can use categories that are better suited to what you are doing. One part can be in several categories, this usually settles itself in the iterative process between analysis/design and coding.
Then, during design, it is put back together and one can see, whether the original problem is still described. Sometimes at this stage it shows itself to be a really great description (the famous "ahah!-effect"), sometimes it just works (which is all we wanted).
- For the design of the system the parts of the problem are organized further into a structure. One idea is to start with a hierarchical structure. If another one is better suited, it will show up due to inconsistencies in the designed system.
- An example for hierarchical organization of the problems is:
overall system level (reason for existence of problem, Requirements, computer system, operating system, interfaces to other systems, etc.)
- high-level software level (what must the software do for system level, user and file I/O, etc.)
unit level (actual programming structures to implement the software level, variables, function parameters, interfaces) Traditional program documentation is done here as a description of the system. Tools are Flow Charts and other diagrams, Program Design Language (PDL), and well commented source code (even before coding).
- It is NOT important to fit everything into the design structure (e.g. whether it's a tree or a line, etc.) it is important instead to watch out for inconsistencies in the design itself, as they reflect such in the software.
Analysis is followed by Synthesis. The decomposed, classified and organized problem's parts are combined into something new (the solution to the problem in software). Again this is usually not a one step process, but at least a very systematical one. Step by step the design of the software to solve the problem is put together, following the methods and concepts described in the next chapters. Thus software that works is created.
Chapter 7: Requirements
- Requirements in the conventional sense is a document containing the things the software should do. It is kept unchanged as a reference for the development process. Thus it is often neglected or abandoned.
- Requirements should not become a dead document but be as flexible as the rest of the software development process. Thus it can be a source of inspiration, a reminder or To-Do-list and integral component of the development process.
- Collect everything you and your customers think would be nice to have, would simplify their work or in any way desireable. The goal is to have a really extensive list at this stage.
- Record what the software will do, how often and the associated time limits. Determine the benefit of each item (gain/loss), then determine the cost (in terms of programming time, computing resources, ...)
- For convenience they should be maintained in a database. Thus it is easy to update the information throughout the project and keep the overview over all tasks, their benefit and cost. Also various reports can be derived from a database.
- These information may change, as the project progresses. New input from the users/client and developers is highly likely.
- Requirements usually result in certain features of the program. It is very useful to understand the relation between both in both directions. Thus you can investigate the effect of changing a feature on your requirements and vice verser.
- Strive for a set of requirements that will lead to a working system. Don't use it to defend non-working software, just because it fulfills all the requirements agreed up front (in the conventional meaning).
- Update the database as soon as a requirement changes, it is critical to be a living picture of the system that you are building. Registering changes per day or even per minute is adequate. Get the requirements into the database, additional information (categories, sorting, ...) is of secondary importance.
- Instead of deleting requirements they should be marked as obsolete, in that way they can be easily reactivated, if the situation changes.
- There is some effort involved in getting into the habit of maintaining requirements, but the amount of work is minimal and the payoff enormous. Besides it is a major part of documentation, as it describes what the software shall and finally will do.
Chapter 8: Decomposition
- Software that does something interesting is usually complicated, because the tasks done by a computer must be described in great detail.
- However, most humans can only cope with a certain level of complexity. Otherwise a high level of concentration is needed, which is often difficult to maintain.
The way out is decomposition, breaking up the complex system into less complex ones. It is an essential part of Analysis and thus design.
- Technically decomposition is the step which describes the requirements in a way understandable for the computer, that is to say in terms of functions, data flow, control flow, interfaces and algorithms (see following chapters).
- This will be done as a functional breakdown in layers of abstraction (hierarchical organization), where the lowest layer for software developers is the actual code. Lower layers are handled by the operating system or the hardware and not necessary to deal with for software development.
These layers start off with some raw categories (for instance Input, Process, Output, see Analysis) and become more detailed in the next layers, e.g. for Output: Open Device, Format Data, Ship Data. In the next level for Ship Data: Fetch Line, Output Line, Advance Line; and so on, right down to the unit code level. For a real system this scheme can become very detailed and is likely not to fit in all on a single sheet of paper. The choice of three items per parent is arbitrary.
- Many requirements are not functional requirements and do not appear in a functional breakdown. Requirements concerning how and where a system must be, financial requirements, performance and interface issues are such. for this reason not every requirement relates to a "leaf" in the functional decomposition tree.
- For the functional requirements make a separate extensive list. At some point of functional breakdown list and tree will converge and the system fits nicely together.
- If it doesn't (typical case) you can do a graphical representation of the entire tree, pin it to a large surface (wall, board) and study the system in all detail (take one or two days). Change the system as you think fit. This is the point, where you really get to know the system and make it your own. This degree of understanding and identification is crucial for creating a working system.
- Besides the relation to the system will usually not end after its completion but continue afterwards, so gaining a thorough understanding is a good thing. It's also more fun to not just have a superficial connection to the system if you have to live with it.
Chapter 9: Dataflow
Chapter 10: Data Dictionary
Chapter 11: The State Machine
Chapter 12: Modularity
Chapter 13: Data Hiding
Chapter 14: Interfaces
Chapter 15: The Module Tree
Chapter 16: Flow Charts
Chapter 17: Conventional Wisdom
Chapter 18: Program Design Language (PDL, Pseudocode)
- PDL is a tool of the design phase, as such it shall not compete for time and energy to write code
- PDL can be easily misunderstood and thus misused, leading to a waste of energy for coding and thus become painful and boring.
- If used correctly it's a very useful tool. So use it correctly!
Example code (actual source code):
Low level PDL:
for each value of i from zero to ten, incrementing i by one, copy the ith element of the array k into the ith element of array j;
Higher level PDL:
copy the first ten elements of k into j
- The last example is probably the lowest sensible level of PDL (if technicalities of an algorithm need to be expressed), otherwise one could write code directly (which is not part of design).
- Mostly even this level will vanish in a more abstract PDL level, describing the step, these ten elements are copied for in the first place.
- PDL should be used directly inside the source code files in shape of a comment of what is done/shall be done in a certain section of code.
- Thus it can help to layout the code before it is typed (implementing the design) and to get an overview in later design cycles.
- This does not exclude to have other useful comments in the source code!
- Plan and prepare the use of PDL, before you start the project, later there will be no time:
- Agree on how pseudo code is written, so it will be understandable when extracted from the source code comments (e.g. how to mark subroutines, ...)
- Work out a procedure to extract only the pseudo code lines from the source file, using special delimiters (like // PS:) and tools like grep or doxygen.
Chapter 19: Error Checking
Chapter 20: Writing Code
Chapter 21: Testing
Chapter 22: Designing with Large Projects and Small
Chapter Last: The Sound of One Hand Clapping
Appendix A: Small Design Example