Part four of the NewsForge's series on system administration is ready for viewing. See part three here.
The role of system administrator is a role of details. Heavily used and updated servers are filled with details, from new tables in a database to root password changes. These details need to be documented. When you are managing three servers, these details can be easy enough to remember. However, when you have 30 or 50 or 100 servers, the details become impossible to keep track of without documenting them. When it matters, you don't want to think that the IP address of that old accounting server is 192.168.10.55, you want to know it.
IV. Thou shalt keep server logs on everything
The only way to ensure that you know the details of all your servers all of the time is to write them down! At my current job, I take care of this using a simple OpenOffice.org spreadsheet. I have four racks of servers, so I use one sheet per rack to track hardware platforms, IP addresses, DNS names, dates of last update and patch installs, operating system, running services, open ports, and other aspects of the machines. It took a day to set it up and get it right, and I update it as I touch servers for various reasons.
