|
|
| November 2000 | Get BSD | New to BSD? | Search BSD | Submit News | FAQ | Contact Us | Join Us |
|
NOTE: This is the first chapter in a series of observations, representing the adventures of a couple of BSD admins (one with a lot of prior MacOS experience, the other with more on the BSD side) poking around the command line on an iBook laptop running Apple's Mac OS X Public Beta. We'll attempt to provide a few notes and observations that may make a BSD admin's work with Mac OS X easier.
Right up front, I should mention what a strange animal Mac OS X is. Since 1984, the Mac OS has always been about the concept that "my OS should work easily, without my knowing how it works." Unix has always been about the idea that "I should be able to control every aspect of my OS, even if it isn't always easy to figure out how." Apple - whose very name is anathema to many Unix admins - is now trying to combine both, in the world's first BSD-based OS that is likely to ship 500,000 copies in its first year.
Apple isn't alone in this hybridization effort; Microsoft will presumably ship "Whistler," its NT/2000-based consumer OS, sometime in mid-2001. It appears that everywhere, the rush is on to deliver consumer OSes based on the kernels of server operating systems. In the long run, this is probably a Good Thing®, but Mac OS X and MS Whistler will have to introduce millions of desktop users to the advantages and problems of server OS design that we as server OS admins have been dealing with for years. Real protected memory and pre-emptive multitasking, sure ... but also real multi-user systems and all of the permission and driver complexities that requires, as well. However, both Microsoft and Apple are investing the time and effort into developing a real user-friendly interface that your grandmother could configure (sorry, KDE and GNOME). It should be interesting to watch.
If you, like most other Internet server admins with a fully-functional brain stem, prefer Unix over the NT kernel, then Mac OS X will be the first true consumer OS that you will ever feel comfortable with. And, whether you love or hate Apple, it's worth your time to get acquainted with Mac OS X and what it offers to a BSD Unix administrator.
Right now, there are a lot of questions out there about Mac OS X and its attempts to marry a BSD / Mach microkernel with an all-singing, all-dancing Mac GUI. This is largely because there is unfortunately a comparatively small crossover between Mac administrators and Unix administrators (much like the slim crossover between supermodels and serious "Dungeons and Dragons" players). Though neither Jeff nor Matt is a total über-guru in Mac OS or BSD, we'll attempt to bridge the gap a little and give the average BSD admin an idea of what he or she will see if they peek around under the hood of Mac OS X Public Beta.
Much of this is "first impression" notes and questions, without too much outside research behind it; explanations or clarifications from those who have more detailed answers are gratefully appreciated. Please send updates or corrections to me@schnell.net or matt@loschert.com .
On a side note, Matt and Jeff will both use "I" rather than "we" in this article as we go, since we are both members of the FreeBSD Borg Collective and think of ourselves as one entity. ;-)
To examine the hidden Unix world on OS X, navigate in the GUI to Applications >> Utilities >> Terminal . The first thing you may notice is that - as you might expect - the default shell is tcsh. You'll quickly find that many of the user apps you'd expect in a typical Unix installation are there, plus a few extras and minus a few others (showing the refreshing influence of many developers, each pushing for the inclusion of their favorite tools). Interestingly, pico is there, although pine is not; Emacs and vi are there, also (ed may be there, but I didn't check since I thought everyone who might use ed was fossilized and hanging on display in a museum by now). Other fun inclusions are gawk, biff, formail, less, locate, groff, perl 5.6, wget, procmail, fetchmail, and an openssh-based client and server.
The first thing that I do when beginning to work on a new system is to set my shell and copy over my personal config files. When presented with the Mac OS X shell prompt, I was pleased to see that it was tcsh; but at the same time I was a little confused at how to customize it. Usually, I just drop my .cshrc file in ~/ , and off I go. Well, ~/ in this case is /Users/myusername . This is unlike the normal /usr/home or /home that most admins are used to. Would tcsh be able to find my .cshrc there?
Well, since I didn't have the machine network-connected while I was originally exploring, and I was too lazy to hand-copy parts of my personal .cshrc file over to this directory, I instead began poking around /etc looking at how the default .cshrc worked. I quickly found a small csh.cshrc which contained a single line sourcing /usr/share/init/tcsh/rc . This is where it gets interesting.
This rc file sets up a very cool method of providing standard system shell features that can be overridden by each Mac OS X user (assuming he/she realizes that the "shell" does not refer to the funky computer case). The rc file first checks to see if the user has a ~/Library/init/tcsh directory. If so, it sets this as the tcsh initialization directory; otherwise it sets the /usr/share/init/tcsh directory as the default. It then proceeds to source other files in the default directory including, for instance, environment, aliases, and completions files which each in turn source files (if they exist) found in the above mentioned tcsh initialization directory.
In this way, the system provides some powerful "standard" features, but still gives the experienced user the ability to override anything and everything. I dropped some customizations in my personal ~/Library/init/tcsh directory and immediately felt at home. Without a doubt, the old school UNIX curmudgeons will hate the built-in shortcuts, but I still must applaud the developers for making an attempt at providing a useful set of features for the "new" user. No one will ever agree on a standard set, but it's good to see that the average Mac "What is this, some kind of DOS prompt?" user will have a very functional command line, should he/she choose to explore it. I must admit that some of the default completions can be a little annoying, though, when you’re typing quickly.
When you drop to the shell, you'll notice that you're logged in with the "user name" you selected for yourself when you installed the OS. When you create a user during the installation process, you are informed that "you will be an administrator of this computer." Yet, when in the shell, you take a look at /etc/passwd, you'll see no entry for your username. Nonetheless, a “root” user exists, and you can su to root. Even stranger, although you are an "administrator," as long as you're logged in as that user, you still don't have the same permissions of "root." What's going on here? There’s a method to this madness, and it’s called NetInfo – the user and network management scheme Mac OS X inherited for NeXT. We’ll run into NetInfo several more times as we poke around the system, and I’ll try to explain some of the basics of this mysterious creature at the end of the article.
I couldn't resist running dmesg to see what would come up. To my surprise, the output was pretty boring. The most bizarre thing was that a packet filter was enabled, but with a default to open policy and no logging. An open firewall ... why? If they weren't planning to enable firewall rules, why compile packet filtering code into the kernel in the first place? At this point, I felt I had to do at least a little searching for some sign that this had been considered at one point. Well, /etc held no clues, and there was nothing obvious in /var (the only other probable location I could think of). I guess they simply wanted to leave the door conveniently ajar should they choose to go back and reconsider the decision.
When I visit a machine, I’m always curious to check out /etc. This directory usually reveals some interesting information about the operating system and/or the administrator. In this case, while touring the usual suspects, I noticed something that, again, I did not expect. The wheel group listed in /etc/group did not contain any users (what is this, one of those Communist Linux systems?). You typically expect to find root along with at least one administrative user account in /etc/group. In this case, wheel was empty, leaving me to wonder why they even kept the concept, since this functionality had apparently been folded into an alternate privilege manager. As I found out, this functionality is also handled by NetInfo.
While checking out /etc/syslog.conf, I noted that quite a bit of system information gets routed to /dev/console, as one would expect. What was strange was that later while exploring /var/tmp (to look for tell-tale temp files), I found a file named console.log. It appeared to contain a file version of the most recent couple of console messages. I verified that identical (albeit fewer) entries appear when using the GUI-based console viewer. I'm no super OS guru ... but this strikes me as a strange hack. Why not simply send the same information to a standard log file via syslog and have the GUI console app read from that? That's what syslog's there for. Maybe someone else out there can shed some light on this one.
Another oddity was /etc/resolv.conf. It was symlinked to /var/run/resolv.conf, which seemed a little strange. Even more so when /var/run/resolv.conf turned out to be non-existent. Okay ... well, there was no named running (I checked) and the resolver man pages gave no clue. So what was going on here? Apparently something odd, since a quick nslookup responded with the DNS server listed as being the local machine. No named, no resolv.conf, how about /etc/hosts? Well, /etc/hosts had no special magic in it, but it did have a note mentioning that the file is only consulted while the machine is in single-user mode. At all other times, the request is handled by lookupd working through the facilities provided by NetInfo. Hmm … NetInfo was beginning to sound very interesting.
After a bit of roaming around, I became curious as to whether root would have a home directory on the machine. Given the difference in user and administrator handling on Mac OS X, I wasn’t even sure that root would have a home directory. But, lo and behold, there was ~root under /var (or more properly /private/var since it's a symlink) with a set of default files and directories resembling any other user in /Users. None of this should have come as a surprise to me since /etc/passwd contained this information.
While exploring the system, you'll find some interesting services enabled. I was surprised to see the nfs daemon running. Thus, it didn’t faze me when I found the portmap daemon (a service required in order to run nfs) hanging around. Finding nfs active was unexpected, since I would only run it if necessary, due to potential security concerns.
Another surprise was seeing a full-fledged NTP daemon running. It was cool to see this service as a standard part of Mac OS; but I thought it a bit strange though to do continuous high accuracy time synchronization via ntp for a desktop, consumer OS. Why not use ntp's little brother, ntpdate, on a periodic basis for this same functionality? Do Mac OS users really need the accuracy of continuous ntp, when a functionally similar result could be obtained without the security risk of running another network daemon?
The answer is that, for now, "yes they do." Since the introduction of Mac OS 9, Mac users have been given the default option of using NTP to synchronize their desktop clocks via NTP servers, and it appears that Apple wants to give Mac OS X users (presumably in a server setting) the option of running an NTP server for other Macs on a LAN/WAN.
Speaking of network services, a little netstat lovin' showed a couple of other interesting tidbits. The machine was not as quiet network-wise as I had expected. Along with the nfs, portmap, and ntp network activity, I found listening sockets open on ports 752 and 755, with an active connection to port 752 from another local privileged port.
After some man page reading and poking at a few of the running daemons, I was able to narrow things down a little. I found that HUP-ing lookupd (an interface daemon to NetInfo) caused the active connection to be disconnected and reestablished (obvious since the new connection originated from a new port). I also found that HUP-ing nibindd (the NetInfo binder, a sort of librarian for NetInfo) caused the listening sockets to be closed and reappear on new ports. That struck me as quite odd. I would have expected them to re-bind to the same ports. Even with these addresses changing, lookupd somehow knew where to find the ports, as I saw new connections established shortly after the HUP signal was received.
Not knowing much about NetInfo, I was curious why this service was implemented using tcp sockets. I assumed that the service must be distributed, available to and from remote hosts. Otherwise, the developers probably would have implemented this using Unix domain sockets since they are substantially faster and are safer for local-only protocols. Not having much background on NetInfo at the time, I was somewhat puzzled.
Apple's longstanding (since Mac System 1.0 in 1984) policy has been to prevent users from screwing up their systems by making it difficult to access the cool things that they might play with and hose their hard drive. This attitude has extended to OS X, and things like header files, gcc and gdb are not present (although these were included in Apple's earlier "developers-only" previews of Mac OS X), not that the average Mac user would be able to “accidentally” damage their system with gcc.
Not to worry, though; the Mach / BSD portion of Mac OS X is maintained separately as an open-source operating system that Apple calls "Darwin." In fact, a uname -a at the command line in MOSXPB reveals:
Darwin ibook 1.2 Darwin Kernel Version 1.2: Wed Aug 30
23:32:53 PDT 2000; root:xnu/xnu-103.obj~1/RELEASE_PPC Power Macintosh powerpc
You can get the above mentioned developer goodies (and much more) by downloading Darwin or following the CVS snapshots for various packages.
At the root of the filesystem, it's not hard to see that most of what you would expect to find is there somewhere; it may just be in a different place than you expected. The first thing you'll likely notice is that a lot of what's there is being hidden from the user by the MacOS X GUI. The Mac OS has always had its share of hidden files (for volume settings, the desktop database, etc.) However, you'll notice a file in / called ".hidden" which lists the items in / that the GUI hides. The contents of .hidden are:
bin (the Unix directory for binaries needed before /usr is mounted)
cores (a link to /private/cores; a directory - theoretically, a uniform location for placing Unix core dumps)
Desktop DB (the GUI desktop database for files, icons, etc.)
Desktop DF (also for the Mac OS 9 desktop)
Desktop Folder (for items that are shown on the desktop)
dev (the Unix devices directory)
etc (a link to /private/etc; contains normal Unix configuration files and other normal /etc stuff)
lib (listed, but not present in /)
lost+found (the Unix directory for "orphaned" files after an unclean system shutdown. Classic Mac OS users are used to looking in the Trash for temp files saved after a crash or other emergency shutdown)
mach (a link to mach.sym)
mach_kernel (the Mach kernel itself)
mach.sym (listed as a Mach-O executable for PowerPC, but I'm not sure exactly what it does. Any Darwin users out there with more knowledge should correct me.)
private (contains var, tmp, etc, cores and a Mac OS X directory called Drivers, whose relationship to /dev is unclear)
sbin (Unix system binaries traditionally needed before /usr mounts)
SystemFolderX (not present in /; may be a typo or used in the future?)
tmp (a link to /private/tmp; the Unix temporary files directory)
Trash (files selected in the GUI to be deleted)
usr (just about everything in BSD Unix these days ;-) )
var (Unix directory meant once upon a time for "variable" files - mail, FTP, PID files, and other goodies)
VM Storage (the classic MacOS virtual memory filesystem - usually equal to the size of physical memory + swap space)
A brief digression about .hidden: Editing this file and adding any directory or file name will cause the GUI to hide the item. For example, edit this file in the shell and add 'Foo,' create a directory in / with that name, log out and log back in, and it won't be seen.
However, it isn't as simple as it sounds. Strangely, there's an item in / called "TheVolumeSettingsFolder" (a Mac OS 9-related directory) whose name isn't in .hidden, but still doesn't show up in the GUI (probably because it’s hidden under MacOS 9, as well). This indicates that there is more to what's shown and hidden in the GUI than just the ".hidden" file. Also, adding a ".hidden" file to another directory than "/" does not appear to cause files of that name to be hidden in that directory. Furthermore, /.hidden does not prevent files with those names from appearing in lower directories, nor does it hide directories in lower directories when an absolute path is placed in .hidden. Thus, it appears that .hidden is sort of a hack, since the Mac HFS+ filesystem already has mechanisms for making directories or files hidden from a GUI.
Getting back on track: You'll notice that all of the traditional BSD file hierarchies are present in /, although the GUI hides them from the user so that they are unaware that these Unix directory structures exist. Furthermore, some of the directories you see are in fact soft links to unusual places (as mentioned, etc, tmp and var are all links to inside /private; I refuse to speculate why without having several drinks of Scotch first). Also, unlike previous incarnations of the Mac OS, the user is unable to rename the "System" directory from the GUI (more info on this later).
As a brief aside, it should be noted that Apple's great struggle here (and the immensity of their effort should be appreciated) is to combine the classic Mac OS "I can move or rename damn near anything I want" ethos with Unix's "it's named that way for a reason" ethos. How Apple chooses to resolve this dilemma in the final release will speak volumes about their dilemma over (I'm ashamed to admit I've forgotten who put it this way, but it's brilliant) an OS that "the user controls," versus an OS that "allows you to use it." And, let's face it, Unix has always been the latter.
Also, logical or physical volumes other than the boot volume are listed under "/" using their "name." Rather than using a scheme DOS's A: or C: drives or even Windows 9x's "My Computer," Mac users have always been able to name their hard drives or partitions arbitrarily. Each drive or partition thereof was then always mounted and visible on the desktop. If I have a drive in my computer which I've named "Jeff's Drive," you'll see it from the shell as: /Jeff's\ Drive, and all of its file and directory contents are viewable underneath it. Similarly, if a user installs MOSX PB on a drive with Mac OS 9 already installed, at installation time their old files and directory hierarchies are all moved to a directory called "Mac OS 9" in /.
Well, by the time I got this far, I realized that this article wouldn’t really be complete without some discussion of the mysterious NetInfo. I also knew that a little bit of research was in order. While searching, I learned that NetInfo is a distributed configuration database consisting of information that would normally be queried through a half-dozen separate sub-systems on a typical Unix platform. For example, NetInfo encompasses user and group authentication and privilege information, name service information, and local and remote service information, to name just a few.
When the NetInfo system is running (i.e. when the system is not in single-user mode), it supersedes the information provided in the standard /etc configuration files, as well as being favored as an information source by system services, such as the resolver. The Apple engineers have accomplished this by hooking a check into each libc system data lookup function to see if NetInfo is running. If so, NetInfo is consulted, otherwise the standard files or services are used.
The genius of NetInfo is that it provides a uniform way of accessing and manipulating all system and network configuration information. A traditional Unix program can call the standard libc system lookup functions and use NetInfo without knowing anything about it. On the other hand, MacOSX-centric programs may directly talk to NetInfo using a common access and update facility for all types of information. No longer does one have to worry about updating multiple configuration files in multiple formats, then restarting one or more system daemon or daemons as necessary.
The other benefit of the system is that it is designed to be network-aware from the ground up. If information cannot be found on the local system, NetInfo may query upward to a possibly more knowledgeable information host. Thus, NetInfo settings can be set for a central server, and all of these settings (users, network, etc.) can be shared by its client computers. Even better, it also knows how to forward requests to the appropriate traditional services if it does not have the requisite information. It can hook into DNS, NIS, and other well-known services, all without the knowledge of the application making the initial data request.
NetInfo also handles users for the system; this is why /etc/passwd and /etc/group are essentially bare, and adding new users through the GUI makes no changes to either of these files. Why? Because once in multi-user mode, these files are not consulted for user information – it’s all handled by NetInfo. The system default-created users listed in /etc/passwd are also in the NetInfo database – but the reverse is not true if you add users to /etc/passwd.
To see a complete list of users that NetInfo “sees” on the system, drop to the shell and use the niutil (Net Info UTILity) command by typing “niutil –list . /users”. Interestingly, when you use the GUI “Multiple Users” tool to manage user accounts, the “Unix-related” default users (root, www, nobody, unknown, and daemon) are not available to be manipulated; this can only be done via the command line and niutil. Interestingly, this means that even though I might log into the GUI as the user “root,” I can still only change my password via the command line; obviously, Apple is intending that no one will log into the GUI as “root.”
This plays into system administration in an interesting way: as mentioned before, Apple tries to “weld the hood shut” in many areas of the system. Let’s say, for example, that I’m logged into the GUI as my normal user named “foo” with administrative privileges. Can I drag the “System” directory to the trash and delete it, or even rename it? Nope. Why?
When I drop to the shell, I see that’s because I’m logged in as “foo,” and /System is owned by root. The GUI allows me (since NetInfo says my user has admin privileges) to use the GUI tools to edit certain configurations (I’m guessing it acts in a sort of “suid” way for this) that are root-owned – but only the items that the GUI has presented me with access to. Of course, I can get around this by logging into the GUI as “root,” or by su-ing to root at the terminal from any other administrative username; from there, I can rm –rf * all I want. It’s an interesting concept, and probably a good idea – you’ll need to know a little about the system before you can screw it up too bad.
However, NetInfo also appears to have some quirks. It’s interesting to note that the user “root” password is set to the same as the first user that you create (usually when setting up the OS for the first time) with “administrator” privileges. No matter how many additional users you create with “administrator” permissions, the root password will always remain that of the first administrative user created (unless you change it manually).
Strangely, some NetInfo changes don’t seem to take place as they should (at least in the OS X Public Beta release). For example: while logged into the GUI, you can use the “Multiple Users” tool to create a new user. Right away, you are able to log out and log back in as that user (or even drop to a terminal and su to that user). However, if you make changes to your IP address/DNS/etc. settings, you are informed that you must restart for the changes to take effect. Even if this stuff is ‘hard-wired’ into the NetInfo setup, it should only require a re-HUP of NetInfo for this to change, not a restart.
For example: logged in as an administrative user, I made changes to the network settings of the computer and clicked “Apply.” Upon being told that I needed to restart for the changes to take effect, I closed down a couple of other applications, typed “reboot” at the shell (yes, it works, as does shutdown) and restarted the computer. I held down the “v” key at startup, which gives you a familiar Unix bootup sequence (all of the console messages as text rather than the prettier but less informative traditional GUI Mac startup screens). I noticed that during startup, strangely, although NetInfo appeared to load before the system set its other networking options, the old settings were still in effect. Only once the GUI had loaded and I logged in as the administrative user did the new network settings take effect. I’m currently unsure whether this is an issue with NetInfo or an issue with rebooting the system via a shell.
NetInfo is a complex beast, easily worth an article of its own (or several). For something so critical to the system, it seems that the quantity of documentation out there for it is pretty awful. It’s interesting to note that, as befits the NeXT heritage of NetInfo, many of the NetInfo-related man pages are dated 1989.
If you want more information, here are a few tips. I found that reading NetInfo man pages was frustrating. Most of the pages tended to heavily use NetInfo-related terms and concepts with little to no definition. Nevertheless, if interested, check out netinfo(5) (netinfo(3) is simply the API definition), netinfod(8), nibindd(8), and lookupd(8). However, the best information that I found was on Apple’s Tech Info Library at http://til.info.apple.com/techinfo.nsf/artnum/n60038?OpenDocument&macosxs .
Finally, for a great resource on all things Darwin, don't go to Apple's website (http://publicsource.apple.com/projects/darwin/). Instead, go to http://www.darwinfo.org/, and you'll find lots of great stuff, including the excellent "Unofficial Darwin FAQ."
If you don't have access to MacOS X Public Beta but would like to read its man pages to get a better idea of how some of these commands are implemented, you can find a collection of MOSXPB/Darwin man pages online at www.osxfaq.com/man . And, as always, I recommend John Siracusa’s set of articles on Ars Technica (arstechnica.com/reviews/4q00/macosx-pb1/macos-x-beta-1.html ) as the best starting point for understanding the technical underpinnings of the system.
Jeff and Matt hope this little tour has been semi-informative and has raised the curiosity of the few brave or bored souls who have made it to the end of this MacOS X travelogue. See you next time.