Code Quality: The Open Source Perspective

by Diomidis Spinellis

Overview

• Page 26: How can I avoid off-by-one errors?
• Page 143: Are Trojan Horse attacks for real?
• Page 158: Where should I look when my application can't handle its workload?
• Page 256: How can I detect memory leaks?
• Page 309: How do I target my application to international markets?
• Page 394: How should I name my code's identifiers?
• Page 441: How can I find and improve the code coverage of my tests?

Diomidis Spinellis' first book, Code Reading, showed programmers how to understand and modify key functional properties of software. Code Quality focuses on non-functional properties, demonstrating how to meet such critical requirements as reliability, security, portability, and maintainability, as well as efficiency in time and space.

Spinellis draws on hundreds of examples from open source projects--such as the Apache web and application servers, the BSD Unix systems, and the HSQLDB Java database--to illustrate concepts and techniques that every professional software developer will be able to appreciate and apply immediately.

Complete files for the open source code illustrated in this book are available online at: http://www.spinellis.gr/codequality/

Product Details

ISBN-13:	9780768685121
Publisher:	Pearson Education
Publication date:	04/03/2006
Series:	Effective Software Development Series
Sold by:	Barnes & Noble
Format:	eBook
Pages:	608
File size:	7 MB
Age Range:	18 Years

About the Author

Diomidis Spinellis has been developing the concepts presented in this book since 1985, while also writing groundbreaking software applications and working on multimillion-line code bases. Spinellis holds an M.Eng. degree in software engineering and a Ph.D. in computer science from Imperial College London. Currently he is an associate professor in the Department of Management Science and Technology at the Athens University of Economics and Business.

Read an Excerpt

In programming, as in everything else, to be in error is to be reborn.
— Alan J. Perlis

I wish I could start this preface by writing that the book you are holding in your hands is the result of a carefully planned premeditated publishing effort that started with the title Code Reading: The Open Source Perspective and is now being completed with Code Quality. Writing so would, however, be twisting the true facts, adjusting reality to the orderly world we engineers like to see around us. The truth is that Code Quality is mostly the result of a series of fortuitous accidents.

When I signed the contract to publish Code Reading, I had in my hands the outline and a couple of completed chapters. I naively calculated the book's length and the completion schedule, based on the length and effort of the chapters I had already written. Now, if you are writing software for a living, you can probably guess that at the time the manuscript was supposed to have been finished, I had covered just slightly more than half the chapters in the outline and had already used up all the allotted pages. Looking for a respectable exit strategy, I suggested to my editor publishing the material I had completed (minus a chapter on portability) as the first volume of Code Reading and continuing the rest of the work in a second volume.We agreed, and Code Reading got published, received a number of favorable reviews, appeared in the list of the 2004 Software Development Magazine Productivity Awards, and got translated into six other languages.

In Code Reading, by using real-life examples taken out of working, open source projects, I tried to cover most code-related concepts that are likely to appear before a software developer's eyes, including programming constructs, data types, data structures, control flow, project organization, coding standards, documentation, and architectures. My plan for the second volume was to cover interfacing and applicationoriented code, including the issues of internationalization and portability, the elements of commonly used libraries and operating systems, low-level code, domain-specific and declarative languages, scripting languages, and mixed-language systems. However, with Code Reading in the hands of programmers, I now had the benefit of readership opinions. The feedback I received indicated that many were eagerly waiting for the follow-up volume, but a detailed dissection of a device driver (one of the chapters I had left for a subsequent volume) was not the material they had in mind for it. In July 2003, my then editor, Mike Hendrickson, suggested working on a book titled Secure Code Reading. Although it security is an area that interests me as a scientist, I was loath to jump into the security book bandwagon and wrote a corresponding chapter instead. With one chapter on portability and one on security, I could suddenly see the book's theme and title before my eyes. Code Quality would focus on how to read and write software code, focusing on its quality attributes, those also often described as nonfunctional properties.

The nonfunctional properties we can discern from reading a software system's code are associated with the product's nonfunctional requirements: the requirements that are not directly concerned with specific functions delivered by the system but that deal with broader emergent system properties. Some common nonfunctional properties are the various -ilities of a system: reliability, portability, usability, interoperability, adaptability, dependability, and maintainability. Two other significant nonfunctional properties concern the system's efficiency: its performance related to time constraints and its space requirements.

The skill of reading code to discern its nonfunctional properties is crucial for two important reasons. First of all, a failure to satisfy a nonfunctional requirement can be critical, even catastrophic. A system that gets some functional requirements wrong (most software products contain such errors) may well be able to operate in a degraded mode; users can be instructed to avoid using some part of the functionality. On the other hand, errors in nonfunctional properties are often showstoppers: an insecure web server or an unreliable antilock brake system (ABS) are worse than useless. In addition, nonfunctional requirements are sometimes difficult to verify.We cannot write a test case to verify a system's reliability or the absence of security vulnerabilities. Therefore, both the critical nature of nonfunctional properties and the difficulty in verifying them suggest that when dealing with nonfunctional requirements and the corresponding software properties, we need to muster all the help we can get. The ability to associate code with nonfunctional properties can be a powerful weapon in a software engineer's arsenal.

Apart from the different perspective, Code Quality follows the successful recipe of Code Reading: focus on the reading of existing code, deal exclusively with realworld examples taken out of existing open source systems, reference all examples to their source, dissect code with annotated listings, provide meaningful exercises to strengthen the reader's critical ability and skills, identify coding idioms and traps in the text's margin, summarize each chapter's advice in the form of maxims, tie practice with theory in the Further Reading section, and use the Unified Modeling Language (UML) for all diagrams. From that recipe, the most tricky ingredient was my self-imposed rule to avoid toy examples, drawing all code samples from existing open source projects. By following the rule, I often found myself spending hours to find an appropriate example: one that would illustrate the concept I was presenting, while also being understandable and short enough to include in the book. I found this exercise both intellectually simulating and a great way to impose discipline on my writing. Often, while searching for a particular weakness, I encountered other interesting elements worthy of discussion. At other times, my search for an example of a theoretical concept proved fruitless: In those cases, I could then credibly reason that the concept was not important enough in practice to include in the text.

The rationale and motivation behind Code Quality are also the same as those that started Code Reading: The reading of code is likely to be one of the most common activities of a computing professional, yet it is seldom taught as a subject or formally used as a method for learning how to design and program. The popularity of open source software has provided us with a large body of code that we can all freely read and learn from. A primer and reader, based on open source software, can be a valuable tool for improving one's programming abilities. I therefore hope that the existence of the two books will spur interest to include code-reading courses, activities, and exercises in the computing education curriculum so that in a few years, our students will learn from existing open source systems, just as their peers studying a language learn from the great literature.

Content and Supplementary Material

I decided to base the source code examples for Code Quality on the same systems and distributions as those I used in Code Reading. I reasoned that it was important to provide continuity between the two volumes, allowing the reader to see how the same source code can be read to discern the functional, architectural, and design characteristics covered in Code Reading and the nonfunctional characteristics covered in Code Quality.

The code used in this book comes from code snapshots that are now mostly only of historic value. This has, however, provided me with the opportunity to show real security vulnerabilities, synchronization problems, portability issues, misused api calls, and other bugs that were identified and fixed in more recent versions. The code base's age makes it likely that its authors by now either have advanced to management positions where reading books as this one is frowned upon or have an eyesight unable to deal with this book's fonts. These changes conveniently provide me with a free license to criticize code without fear of nasty retributions. Nevertheless, I understand that I can be accused of disparaging code that was contributed by its authors in good faith to further the open source movement and to be improved upon rather than bemerely criticized. I sincerely apologize in advance if my comments cause any offense to a source code author. In defense, I argue that in most cases, the comments do not target the particular code excerpt but rather use it to illustrate a practice that should be avoided. Often the code I am using as a counterexample is a sitting duck, as it was written at a time when technological and other restrictions justified the particular coding practice, or the particular practice is criticized out of the context. In any case, I hope that the comments will be received good-humoredly and openly admit that my own code contains similar, and probably worse, misdeeds.

I chose all the systems used in the book's examples for practical reasons having to do with the suitability of the code as an instructional vehicle. Things I looked for were code quality, structure, design, utility, popularity, and a license thatwould not makemy publisher nervous. I strived to balance the selection of languages, actively looking for suitable Java and C++ code. However, where similar concepts could be demonstrated using different languages, I chose to use C as the least common denominator. Thus, 61% of the code references in the book are to C code; these include examples related to programming in the small (applicable to any language) and systems programming (which is done mostly in C). Another 19% of the examples refer to Java code. I chose to use Java code to demonstrate elements associated with object-oriented concepts and the corresponding apis. Most of these concepts also apply verbatim to C# and many apply to C++ (which is referenced in 4% of the examples).

I've also put more emphasis on Unix apis and tools than on the corresponding Windows elements. My reasoning here also involved the logic of the least common denominator: Many of the Unix tools and apis are also available under Windows, whereas the reverse is not true. Also, a number of Unix-compatible systems, such as gnu/Linux and the bsd variants, are freely available, often in the form of a bootable live cd-rom, so that anyone can easily experiment with such a system. Finally, the Unix apis and tools, at the level of detail I use in my examples, have remained remarkably stable over the past 30 years, providing us with an excellent base for discussing and illustrating general principles. Nevertheless, in a number of places, I reference Windows apis and commands to discuss how things work on a different platform. Don't let those references fool you: I don't pretend that this book's coverage of the Windows platform programming issues is complete or even comprehensive, any more than I claim the same for the Unix systems.

Apart from the use of open source software for all its examples, this book might be accused of (narrowly) missing a number of popular bandwagons, including Java, C#, Windows, Linux, and a writing style oriented toward solving today's itch now. I value all of the preceding: 29% of the machines under my roof run Linux, I teach a Java programming course, I've written a number of programs for the Windows platform, and my bookshelf has at least ten books filled with sequentially numbered paragraphs offering concrete, problem-solving advice. However, I also believe that in today's changing world, it is important to understand the principles behind the homily. As you will see in the following chapters, once we focus on the principles

• The choice of the underlying technology is often immaterial
• What we learn has a wider applicability and a longer lifespan
• The concrete advice comes on its own (look at the list of "advice to take home" at the end of each chapter)

Most important, the understanding of the principles behind our craft is what distinguishes an expendable coder from a valued software engineer.

Table of Contents

List of Tables     xv
List of Figures     xvii
Foreword    xxiii
Preface     xxv

Chapter 1: Introduction     1

1.1 Software Quality      1
1.2 How to Read This Book     9

Chapter 2: Reliability     17

2.1 Input Problems      17
2.2 Output Problems     21
2.3 Logic Problems      26
2.4 Computation Problems      42
2.5 Concurrency and Timing Problems     51
2.6 Interface Problems     56
2.7 Data-Handling Problems      69
2.8 Fault Tolerance     85

Chapter 3: Security     101

3.1 Vulnerable Code     102
3.2 The Buffer Overflow      106
3.3 Race Conditions     112
3.4 Problematic APIs     115
3.5 Untrusted Input     125
3.6 Result Verification     131
3.7 Data and Privilege Leakage      134
3.8 Trojan Horse      143
3.9 Tools      146

Chapter 4: Time Performance     151

4.1 Measurement Techniques      156
4.2 Algorithm Complexity     173
4.3 Stand-Alone Code      179
4.4 Interacting with the Operating System      182
4.5 Interacting with Peripherals     190
4.6 Involuntary Interactions     191
4.7 Caching      194

Chapter 5: Space Performance     207

5.1 Data      209
5.2 Memory Organization      227
5.3 Memory Hierarchies      231
5.4 The Process/Operating System Interface     239
5.5 Heap Memory Management      246
5.6 Stack Memory Management     264
5.7 Code     274

Chapter 6: Portability     289

6.1 Operating Systems     290
6.2 Hardware and Processor Architectures     296
6.3 Compilers and Language Extensions     302
6.4 Graphical User Interfaces      307
6.5 Internationalization and Localization      309

Chapter 7: Maintainability     325

7.1 Measuring Maintainability      326
7.2 Analyzability     351
7.3 Changeability      403
7.4 Stability     418
7.5 Testability      432
7.6 Effects of the Development Environment      451

Chapter 8: Floating-Point Arithmetic     465

8.1 Floating-Point Representation      466
8.2 Rounding      478
8.3 Overflow      481
8.4 Underflow      483
8.5 Cancellation      487
8.6 Absorption      491
8.7 Invalid Operations     495

Appendix A: Source Code Credits     503

Bibliography     505
Index     523
Author Index     563

Preface

In programming, as in everything else, to be in error is to be reborn.
— Alan J. Perlis

I wish I could start this preface by writing that the book you are holding in your hands is the result of a carefully planned premeditated publishing effort that started with the title Code Reading: The Open Source Perspective and is now being completed with Code Quality. Writing so would, however, be twisting the true facts, adjusting reality to the orderly world we engineers like to see around us. The truth is that Code Quality is mostly the result of a series of fortuitous accidents.

When I signed the contract to publish Code Reading, I had in my hands the outline and a couple of completed chapters. I naively calculated the book's length and the completion schedule, based on the length and effort of the chapters I had already written. Now, if you are writing software for a living, you can probably guess that at the time the manuscript was supposed to have been finished, I had covered just slightly more than half the chapters in the outline and had already used up all the allotted pages. Looking for a respectable exit strategy, I suggested to my editor publishing the material I had completed (minus a chapter on portability) as the first volume of Code Reading and continuing the rest of the work in a second volume.We agreed, and Code Reading got published, received a number of favorable reviews, appeared in the list of the 2004 Software Development Magazine Productivity Awards, and got translated into six other languages.

In Code Reading, by using real-life examples taken out of working, open source projects, I tried to cover most code-related concepts that are likely to appear before a software developer's eyes, including programming constructs, data types, data structures, control flow, project organization, coding standards, documentation, and architectures. My plan for the second volume was to cover interfacing and applicationoriented code, including the issues of internationalization and portability, the elements of commonly used libraries and operating systems, low-level code, domain-specific and declarative languages, scripting languages, and mixed-language systems. However, with Code Reading in the hands of programmers, I now had the benefit of readership opinions. The feedback I received indicated that many were eagerly waiting for the follow-up volume, but a detailed dissection of a device driver (one of the chapters I had left for a subsequent volume) was not the material they had in mind for it. In July 2003, my then editor, Mike Hendrickson, suggested working on a book titled Secure Code Reading. Although it security is an area that interests me as a scientist, I was loath to jump into the security book bandwagon and wrote a corresponding chapter instead. With one chapter on portability and one on security, I could suddenly see the book's theme and title before my eyes. Code Quality would focus on how to read and write software code, focusing on its quality attributes, those also often described as nonfunctional properties.

The nonfunctional properties we can discern from reading a software system's code are associated with the product's nonfunctional requirements: the requirements that are not directly concerned with specific functions delivered by the system but that deal with broader emergent system properties. Some common nonfunctional properties are the various -ilities of a system: reliability, portability, usability, interoperability, adaptability, dependability, and maintainability. Two other significant nonfunctional properties concern the system's efficiency: its performance related to time constraints and its space requirements.

The skill of reading code to discern its nonfunctional properties is crucial for two important reasons. First of all, a failure to satisfy a nonfunctional requirement can be critical, even catastrophic. A system that gets some functional requirements wrong (most software products contain such errors) may well be able to operate in a degraded mode; users can be instructed to avoid using some part of the functionality. On the other hand, errors in nonfunctional properties are often showstoppers: an insecure web server or an unreliable antilock brake system (ABS) are worse than useless. In addition, nonfunctional requirements are sometimes difficult to verify.We cannot write a test case to verify a system's reliability or the absence of security vulnerabilities. Therefore, both the critical nature of nonfunctional properties and the difficulty in verifying them suggest that when dealing with nonfunctional requirements and the corresponding software properties, we need to muster all the help we can get. The ability to associate code with nonfunctional properties can be a powerful weapon in a software engineer's arsenal.

Apart from the different perspective, Code Quality follows the successful recipe of Code Reading: focus on the reading of existing code, deal exclusively with realworld examples taken out of existing open source systems, reference all examples to their source, dissect code with annotated listings, provide meaningful exercises to strengthen the reader's critical ability and skills, identify coding idioms and traps in the text's margin, summarize each chapter's advice in the form of maxims, tie practice with theory in the Further Reading section, and use the Unified Modeling Language (UML) for all diagrams. From that recipe, the most tricky ingredient was my self-imposed rule to avoid toy examples, drawing all code samples from existing open source projects. By following the rule, I often found myself spending hours to find an appropriate example: one that would illustrate the concept I was presenting, while also being understandable and short enough to include in the book. I found this exercise both intellectually simulating and a great way to impose discipline on my writing. Often, while searching for a particular weakness, I encountered other interesting elements worthy of discussion. At other times, my search for an example of a theoretical concept proved fruitless: In those cases, I could then credibly reason that the concept was not important enough in practice to include in the text.

The rationale and motivation behind Code Quality are also the same as those that started Code Reading: The reading of code is likely to be one of the most common activities of a computing professional, yet it is seldom taught as a subject or formally used as a method for learning how to design and program. The popularity of open source software has provided us with a large body of code that we can all freely read and learn from. A primer and reader, based on open source software, can be a valuable tool for improving one's programming abilities. I therefore hope that the existence of the two books will spur interest to include code-reading courses, activities, and exercises in the computing education curriculum so that in a few years, our students will learn from existing open source systems, just as their peers studying a language learn from the great literature.

Content and Supplementary Material

I decided to base the source code examples for Code Quality on the same systems and distributions as those I used in Code Reading. I reasoned that it was important to provide continuity between the two volumes, allowing the reader to see how the same source code can be read to discern the functional, architectural, and design characteristics covered in Code Reading and the nonfunctional characteristics covered in Code Quality.

The code used in this book comes from code snapshots that are now mostly only of historic value. This has, however, provided me with the opportunity to show real security vulnerabilities, synchronization problems, portability issues, misused api calls, and other bugs that were identified and fixed in more recent versions. The code base's age makes it likely that its authors by now either have advanced to management positions where reading books as this one is frowned upon or have an eyesight unable to deal with this book's fonts. These changes conveniently provide me with a free license to criticize code without fear of nasty retributions. Nevertheless, I understand that I can be accused of disparaging code that was contributed by its authors in good faith to further the open source movement and to be improved upon rather than bemerely criticized. I sincerely apologize in advance if my comments cause any offense to a source code author. In defense, I argue that in most cases, the comments do not target the particular code excerpt but rather use it to illustrate a practice that should be avoided. Often the code I am using as a counterexample is a sitting duck, as it was written at a time when technological and other restrictions justified the particular coding practice, or the particular practice is criticized out of the context. In any case, I hope that the comments will be received good-humoredly and openly admit that my own code contains similar, and probably worse, misdeeds.

I chose all the systems used in the book's examples for practical reasons having to do with the suitability of the code as an instructional vehicle. Things I looked for were code quality, structure, design, utility, popularity, and a license thatwould not makemy publisher nervous. I strived to balance the selection of languages, actively looking for suitable Java and C++ code. However, where similar concepts could be demonstrated using different languages, I chose to use C as the least common denominator. Thus, 61% of the code references in the book are to C code; these include examples related to programming in the small (applicable to any language) and systems programming (which is done mostly in C). Another 19% of the examples refer to Java code. I chose to use Java code to demonstrate elements associated with object-oriented concepts and the corresponding apis. Most of these concepts also apply verbatim to C# and many apply to C++ (which is referenced in 4% of the examples).

I've also put more emphasis on Unix apis and tools than on the corresponding Windows elements. My reasoning here also involved the logic of the least common denominator: Many of the Unix tools and apis are also available under Windows, whereas the reverse is not true. Also, a number of Unix-compatible systems, such as gnu/Linux and the bsd variants, are freely available, often in the form of a bootable live cd-rom, so that anyone can easily experiment with such a system. Finally, the Unix apis and tools, at the level of detail I use in my examples, have remained remarkably stable over the past 30 years, providing us with an excellent base for discussing and illustrating general principles. Nevertheless, in a number of places, I reference Windows apis and commands to discuss how things work on a different platform. Don't let those references fool you: I don't pretend that this book's coverage of the Windows platform programming issues is complete or even comprehensive, any more than I claim the same for the Unix systems.

Apart from the use of open source software for all its examples, this book might be accused of (narrowly) missing a number of popular bandwagons, including Java, C#, Windows, Linux, and a writing style oriented toward solving today's itch now. I value all of the preceding: 29% of the machines under my roof run Linux, I teach a Java programming course, I've written a number of programs for the Windows platform, and my bookshelf has at least ten books filled with sequentially numbered paragraphs offering concrete, problem-solving advice. However, I also believe that in today's changing world, it is important to understand the principles behind the homily. As you will see in the following chapters, once we focus on the principles

• The choice of the underlying technology is often immaterial
• What we learn has a wider applicability and a longer lifespan
• The concrete advice comes on its own (look at the list of 'advice to take home' at the end of each chapter)

Most important, the understanding of the principles behind our craft is what distinguishes an expendable coder from a valued software engineer.