eBook
Available on Compatible NOOK devices, the free NOOK App and in My Digital Library.
Related collections and offers
Overview
Mac OS X was released in March 2001, but many components, such as Mach and BSD, are considerably older. Understanding the design, implementation, and workings of Mac OS X requires examination of several technologies that differ in their age, origins, philosophies, and roles.
Mac OS X Internals: A Systems Approach is the first book that dissects the internals of the system, presenting a detailed picture that grows incrementally as you read. For example, you will learn the roles of the firmware, the bootloader, the Mach and BSD kernel components (including the process, virtual memory, IPC, and file system layers), the object-oriented I/O Kit driver framework, user libraries, and other core pieces of software. You will learn how these pieces connect and work internally, where they originated, and how they evolved. The book also covers several key areas of the Intel-based Macintosh computers.
A solid understanding of system internals is immensely useful in design, development, and debugging for programmers of various skill levels. System programmers can use the book as a reference and to construct a better picture of how the core system works. Application programmers can gain a deeper understanding of how their applications interact with the system. System administrators and power users can use the book to harness the power of the rich environment offered by Mac OS X. Finally, members of the Windows, Linux, BSD, and other Unix communities will find the book valuable in comparing and contrasting Mac OS X with their respective systems.
Mac OS X Internals focuses on the technical aspects of OS X and is so full of extremely useful information and programming examples that it will definitely become a mandatory tool for every Mac OS X programmer.
Product Details
| ISBN-13: | 9780132702263 |
|---|---|
| Publisher: | Pearson Education |
| Publication date: | 06/19/2006 |
| Sold by: | Barnes & Noble |
| Format: | eBook |
| Pages: | 1680 |
| File size: | 81 MB |
| Note: | This product may take a few minutes to download. |
About the Author
Table of Contents
Preface xxv
Acknowledgments xxxi
About the Author xxxiii
Chapter 1 Origins of Mac OS X 1
1.1 Apple's Quest for the Operating System 2
1.2 The NeXT Chapter 9
1.3 The Mach Factor 15
1.4 Strategies 24
1.5 Toward Mac OS X 31
Chapter 2 An Overview of Mac OS X 432.1 Firmware 46
2.2 Bootloader 46
2.3 Darwin 46
2.4 The xnu Kernel 48
2.5 A User-Space View of the File System 57
2.6 The Runtime Architecture 61
2.7 The C Library 74
2.8 Bundles and Frameworks 76
2.9 Core Services 89
2.10 Application Services 90
2.11 Application Environments 101
2.12 User Interface 117
2.13 Programming 121
2.14 Security 131
2.15 Mac OS X Server 145
2.16 Networking 153
Chapter 3 Inside an Apple 1553.1 The Power Mac G5 156
3.2 The G5: Lineage and Roadmap 166
3.3 The PowerPC 970FX 174
3.4 Software Conventions 224
3.5 Examples 240
Chapter 4 The Firmware and the Bootloader 2634.1 Introduction 263
4.2 A Whole New World 266
4.3 Power-On Reset 271
4.4 Open Firmware 272
4.5 Forth 279
4.6 The Device Tree 289
4.7 Open Firmware Interfaces 298
4.8 Programming Examples 300
4.9 Firmware Boot Sequence 324
4.10 BootX 328
4.11 Alternate Booting Scenarios 340
4.12 Firmware Security 349
4.13 Launching the Kernel 352
4.14 The BootCache Optimization 353
4.15 The Boot-Time Kernel Arguments 355
4.16 The Extensible Firmware Interface 362
Chapter 5 Kernel and User-Level Startup 3815.1 Arranging for the Kernel to Execute 382
5.2 Low-Level Processor Initialization 388
5.3 High-Level Processor Initialization 405
5.4 Mach Subsystem Initialization 421
5.5 The First Thread 432
5.6 I/O Kit Initialization 435
5.7 BSD Initialization 443
5.8 Launching the First User-Space Program 469
5.9 Slave Processors 470
5.10 User-Level Startup 472
Chapter 6 The xnu Kernel 5016.1 xnu Source 501
6.2 Mach 510
6.3 A Flavor of the Mach APIs 519
6.4 Entering the Kernel 529
6.5 Exception Processing 543
6.6 System Call Processing 553
6.7 System Call Categories 557
6.8 Kernel Support for Debugging, Diagnostics, and Tracing 601
6.9 Virtual Machine Monitor 659
6.10 Compiling the Kernel 676
Chapter 7 Processes 6837.1 Processes: From Early UNIC to Mac OS X 684
7.2 Mach Abstractions, Data Structures, and APIs 687
7.3 Many Threads of a New System 726
7.4 Scheduling 774
7.5 The execve() System Call 812
7.6 Launching Applications 828
Chapter 8 Memory 8358.1 Looking Back 835
8.2 An Overview of Mac OS X Memory Management 838
8.3 Mac VM 846
8.4 Resident Memory 868
8.5 Virtual Memory Initialization during Bootstrap 877
8.6 The Mach VM User-Space Interface 878
8.7 Using the Mach VM Interfaces 893
8.8 Kernel and User Address Space Layouts 907
8.9 Universal Page Lists (UPLs) 912
8.10 Unified Buffer Cache (UBC) 913
8.11 The Dynamic Pager Program 918
8.12 The Update Daemon 921
8.13 System Shared Memory 922
8.14 Task Working Set Detection and Maintenance 942
8.15 Memory Allocation in User Space 948
8.16 Memory Allocation in the Kernel 980
8.17 Memory-Mapped Files 1001
8.18 64-bit Computing 1005
Chapter 9 Interprocess Communication 10219.1 Introduction 1021
9.2 Mach IPC: An Overview 1025
9.3 Mach IPC: The Mac OS X Implementation 1041
9.4 Name and Bootstrap Servers 1060
9.5 Using Mach IPC 1080
9.6 MIG 1094
9.7 Mach Exceptions 1112
9.8 Signals 1129
9.9 Pipes 1145
9.10 Named Pipes (Fifos) 1147
9.11 File Descriptor Passing 1148
9.12 XSI IPC 1155
9.13 POSIX IPC 1156
9.14 Distributed Objects 1164
9.15 Apple Events 1172
9.16 Notifications 1181
9.17 Core Foundation IPC 1197
9.18 Synchronization 1210
Chapter 10 Extending the Kernel 123310.1 A Driver down the Memory Lane 1233
10.2 The I/O Kit 1235
10.3 DART 1257
10.4 Dynamically Extending the Kernel 1259
10.5 Communicating with the Kernel 1269
10.6 Creating Kernel Extensions 1271
10.7 A Programming Tour of the I/O Kit's Functionality 1288
10.8 Debugging 1321
Chapter 11 File Systems 134511.1 Disks and Partitions 1345
11.2 Disk Arbitration 1353
11.3 The Implementation of Disk Devices 1362
11.4 Disk Images 1366
11.5 Files and File Descriptors 1374
11.6 The VFS Layer 1376
11.7 File System Types 1386
11.8 Spotlight 1409
11.9 Access Control Lists 1441
11.10 The Kauth Authorization Subsystem 1445
Chapter 12 The HFS Plus File System 147112.1 Analysis Tools 1474
12.2 Fundamental Concepts 1477
12.3 The Structure of an HFS+ Volume 1491
12.4 Reserved Areas 1493
12.5 The Volume Header 1493
12.6 The HFS Wrapper 1501
12.7 Special Files 1505
12.8 Examining HFS+ Features 1531
12.9 Optimizations 1558
12.10 Miscellaneous Features 1570
12.11 Comparing Mac OS X File Systems
12.12 Comparing HFS+ and NTFS 1582
Appendix A Mac OS X on x86-Based Macintosh Computers 1587A.1 Hardware Differences 1587
A.2 Firmware and Booting 1589
A.3 Partitioning 1590
A.4 Universal Binaries 1591
A.5 Rosetta 1592
A.6 Byte Ordering 1594
A.7 Miscellaneous Changes 1594
Index 1599Preface
Although Mac OS X is a relatively new operating system, its lineage is rather colorful, and the history of most of its components is much older. Mac OS X, and Apple in general, has attracted a lot of attention in recent years. Given Apple's "cult-like" status, and the unique cultural and technical composition of Mac OS X, it is not surprising that people with vastly different backgrounds and interests—both users and non-users of Mac OS X—are curious about the system.
After working on, programming on, and extending several operating systems over the years, I was introduced to Mac OS X on April 1, 2003. I was soon curious about the structure of the system. Despite the availability of several good books on Mac OS X, I was dismayed to find that I could not learn the details of Mac OS X internals from a book—no such book existed. There are books describing how to perform various tasks on Mac OS X; how to configure, customize, and tweak the system; and how certain user-visible aspects differ between Mac OS X and Windows. There are also books covering specific Mac OS X programming topics, such as the Cocoa and Carbon APIs. Other books make life easier for a Unix user migrating to Mac OS X—such books typically discuss the command-line interface to the operating system. Although these books play important roles in making Mac OS X accessible, the core architecture and implementation of Mac OS X and its components have remained mysteries. To make matters worse, besides the lack of information, it is common to find misinformation on the composition of Mac OS X. Consequently, the system is often misunderstood because of long-standing myths and stereotypes, or is perceived as a black box.
The purpose of this book is to deconstruct and demystify Mac OS X by describing it from a system-design perspective. It takes an implementation-oriented approach to understanding the system. Consider the example of interprocess communication (IPC). Mac OS X has numerous user-level and kernel-level IPC mechanisms, some of which are well known and documented. Rather than just showing how to use these mechanisms, the book explains the design and implementation of the most fundamental mechanism (Mach IPC), and then discusses how the other mechanisms are layered on top of one another. My goal is not to teach you how to do something specific—my goal is to provide you with enough knowledge and examples so that after reading the book, depending on your interests and background, you can build upon your newly gained knowledge and do what you choose.
Along with text, the book uses detailed illustrations, function call graphs, annotated code excerpts, and programming examples to present a detailed examination of Mac OS X. To keep the subject matter interesting and accessible—even to the casual reader—the book contains relevant trivia, digressions, and other tidbits.
Who This Book Is For
I hope the book will be useful to anyone curious about the composition and working of Mac OS X.
Application programmers can gain a deeper understanding of how their applications interact with the system. System programmers can use the book as a reference and to construct a better picture of how the core system works. In my experience as a programmer, a solid understanding of system internals is immensely useful in design, development, and debugging. For example, you know what the system is capable of, what is feasible and what is not, what the "best" option is in a given situation, and what the plausible reasons are for certain program behavior. This book's primary goal is to build a strong foundation for anyone who programs on Mac OS X.
Mac OS X users can read the book to better understand how the system is designed and how it comes together. System administrators and technical support staff will also find value in the book.
Besides those who use Mac OS X, the intended audience includes members of other technical communities, such as the BSD, Linux, and Windows communities. Given that many internal aspects of Mac OS X are radically different from these systems (for example, how the Mach kernel is used), the book will help such readers broaden their knowledge, and will assist them in comparing and contrasting Mac OS X with other operating systems.
The book will also be useful in an advanced operating systems course, particularly if you wish to do a case study on Mac OS X. The book is not suitable, however, as an introductory text. Most of the material is not presented at an introductory level, although I introduce many of the advanced topics with at least some background information.
The Book's Structure
Modern operating systems have become so large and complex that it is impossible to reasonably describe an entire system in a book. This book is somewhat ambitious in that it attempts to cover Mac OS X in substantial breadth and depth. The most important contributors to the book's depth are the carefully selected programming examples. The book is organized into twelve chapters. Although much of the book's content is rather technical, each chapter has sections that should be accessible to non-programmers.
Chapter 1, "Origins of Mac OS X," describes the technical history of Mac OS X and the systems it derives from. An unabridged version of Chapter 1, which covers all of Apple's past and present operating systems, is available on this book's accompanying web site.
Chapter 2, "An Overview of Mac OS X," is an eclectic tour of Mac OS X and its important features. It contains brief overviews of various layers that constitute the system.
Chapter 3, "Inside an Apple," describes the PowerPC architecture, using the PowerPC 970 ("G5") processor line as a specific example. It also discusses the PowerPC assembly language and calling conventions.
Chapter 4, "The Firmware and the Bootloader," describes both Open Firmware and the Extensible Firmware Interface (EFI), along with their respective boot-loaders. It discusses the roles the firmware and the bootloader play in the system's operation, usage scenarios, and events that occur during early bootstrapping.
Chapter 5, "Kernel and User-Level Startup," describes the sequence of events—including initializations of kernel subsystems—from where the kernel starts executing to the point where the first user-space program (launchd) is run by the kernel. The discussion includes launchd's function and implementation.
Chapter 6, "The xnu Kernel," describes the core kernel architecture of Mac OS X. The discussion includes system call families and their implementation, low-level tracing and debugging mechanisms, and special features such as the virtual machine monitor in the PowerPC version of the kernel.
Chapter 7, "Processes," describes abstractions such as tasks, threads, and processes, the various forms in which they exist in Mac OS X subsystems, and processor scheduling. The discussion includes using various kernel-level and user-level interfaces for manipulating the aforementioned abstractions.
Chapter 8, "Memory," describes the Mac OS X memory subsystem's architecture, including discussions of the Mach virtual memory architecture, paging, the unified buffer cache, the working-set detection mechanism, kernel-level and user-level memory allocators, and support for 64-bit addressing.
Chapter 9, "Interprocess Communication," describes various IPC and synchronization mechanisms available in Mac OS X. In particular, it discusses the implementation and usage of Mach IPC.
Chapter 10, "Extending the Kernel," describes the I/O Kit, the object-oriented driver subsystem in Mac OS X.
Chapter 11, "File Systems," describes the overall file system layer in Mac OS X, including brief discussions of each file system type. The discussion also covers partitioning schemes, disk management, and the Spotlight search technology.
Chapter 12, "The HFS Plus File System," describes the internals of the HFS Plus file system. The discussion is aided by the use of a custom file system debugger written for this chapter.
Appendix A, "Mac OS X on x86-Based Macintosh Computers," highlights the key differences between the x86-based and PowerPC-based versions of Mac OS X. Besides this appendix, the book covers the details of several key x86-specific topics, such as EFI, GUID-based partitioning, and Universal Binaries. Most of Mac OS X is architecture-independent, and consequently, the majority of the book is architecture-independent.
Given the book's length, I chose to exclude several topics that are well covered in other texts. The TCP/IP stack is an example—there is no "networking" chapter in the book since the Mac OS X TCP/IP stack is largely a derivative of the FreeBSD stack, which is already well documented. In general, information that is generic across Unix variants and can be found in standard texts is not included in this book.
How to Read This Book
Since the first two chapters provide the background and overall picture of Mac OS X, respectively, I recommend that you read these chapters first. The subsequent chapters are best read sequentially, although, depending on your interests and familiarity with the topics, you can skip certain sections (and perhaps even chapters) and still gain value from the book.
It will help if you have some familiarity with operating system concepts and have used a Unix operating system.
Given that the book has a large number of C programs and program excerpts, you should have some programming experience, and in particular, knowledge of the C programming language. I sometimes use code not only to demonstrate the working of a concept, but also to describe the concept. I realize that it is usually considered "difficult" to "read" code, and authors often expect that many readers will simply skip the code. My belief is that reading the code (as opposed to only running it) in this book will be particularly helpful to programmers.
Despite the book's technical nature, several parts of the book can be read casually by both programmers and non-programmers. I hope that as a reference on Mac OS X internals, the book and its examples will be useful to its readers for a long time to come.
How to Use the Examples
I have included a number of self-contained examples in the book. Many of these examples are non-trivial in that they do something that is both useful and interesting. I hope that these examples will serve as food for thought and building blocks for other projects. Almost all of the examples in the book are shown along with the command lines used to compile and run them.
The examples were tested on both PowerPC-based and x86-based Macintosh computers where applicable. It is interesting to note that in the cases where the code is PowerPC-only, say, in a PowerPC assembly language example, it can usually be both compiled and run on an x86-based Macintosh—such code will run under the Rosetta binary translation software. However, a small number of examples in the book require a PowerPC Macintosh—they will not run under Rosetta.
Related Material
Technology moves so fast these days that it is almost impossible to publish a fully up-to-date book. Thankfully, Internet access allows the author and publisher to make various materials available to readers after the book is published. The most useful resource for this book is its accompanying web site, http://www.osxbook.com, which includes the following:
- Errata and updates
- Source code from the book
- The book's blog, with news and announcements about the availability of new material
- A set of discussion forums where topics related to the book (and to Mac OS X in general) can be discussed
- A bonus content section, which contains additional articles, presentations, binaries, and source code relevant to the book
- Sample content from the book, including a detailed table of contents