Wednesday, July 23, 2014

The road to Reposado

One of the projects that I've had in the back of my head for a while was to ditch Apple's Software Update Service (SUS) and use Reposado in its place.  We currently have several (20) servers in place. During our ongoing evaluation process we've begun to question the necessity of maintaining these servers in their current capacity. We are using them for 3 purposes: Workgroup Manager (MCX), a Casper (JSS) Distribution Point, and Software Update Services.
  • MCX is being replaced by profiles.  As we move more of our computers to 10.8/10.9, it is no longer necessary to continue this service.
  • We could use (virtually) any server to host our Distribution Points.
  • Software Update Services only (officially) hosts updates for the same level of the host's operating system or lower. Since our servers are currently at 10.7.5 this obviously creates a problem.  We were able to use a kludge to get it to host 10.8.X updates, but this didn't work when 10.9 was released.
So, as you can see, we can possibly do without maintaining Apple branded servers in our environment. As an experiment, on a lonely Sunday afternoon, I decided to install Jamf's Netboot/SUS appliance in our virtual infrastructure. A couple hours and a few hairs later I was servering updates back through 10.4.X.  I quietly put this into production and began servicing our 10.9 clients. This would normally be fine, except that we have relatively slow links to our remote sites

Thus this past week I launched my master plan to replace Apple's SUS with Reposado.  Below is the process that I used. (Warning: This is the process that I used.  I'm sure that there are many other ways to do what I've done.)

Resources used:


Preparation:
  • Clone (download) Reposado and Margarita repositories.
  • Download X-Code command line tools

Process:
Create Reposado working directory:
mkdir /usr/local/reposado

Copy items in reposado-master/code directory to this folder.

Create Reposado repository folders:
mkdir -p /Volumes/Data/reposado/html
mkdir /Volumes/Data/reposado/metadata

Change to the Reposado working directory:
cd /usr/local/reposado

Configure Reposado and answer the questions that come up. (Answers in bold) In this example I’ll be using port 80 to serve updates:
./repoutil --configure

Path to store replicated catalogs and updates [None]: /Volumes/Data/reposado/html
Path to store Reposado metadata [None]: /Volumes/Data/reposado/metadata/
Base URL for your local Software Update Service (Example: http://su.your.org -- leave empty if you are not replicating updates) [None]: http://myupdateserver.com 

Clone the Apple Software Update library:
./repo_sync

The initial sync will take a couple of hours. While this is running, you should see it creating files and indexes and downloading the updates into it’s file structure.  Since this is going to take a while, we can begin to prep our web server.

Open the Apache2 configuration file (/etc/apache2/httpd.conf) to make sure it’s setup properly:

•Make sure it’s listening on port 80 (around line 50?):
#Listen 12.34.56.78:80
Listen 80


•The rewrite module is enabled: (Uncomment if necessary)
LoadModule rewrite_module libexec/apache2/mod_rewrite.so

•Change the default directory. Comment out the default line and add your new directory so that it looks like below:

Change: Directory "/Library/WebServer/Documents"
to: Directory "/Volumes/Data/reposado"

•Some people have found it necessary to make sure that Overrides are enabled (I haven’t found it necessary, but ymmv)
Change: AllowOverride None
to: AllowOverride All

Close the file and edit the default port 80 configuration (/etc/apache2/sites/0000_any_80_.conf) :

Change the default directory here as well:
DocumentRoot "/Volumes/Data/reposado/html”

(This step is probably not necessary, but this is the combination was required in order for me to get this working.)

Create a .htaccess file in your Reposado repository with the following items (check the Reposado wiki for the latest). This will allow you to point all of your clients to [repositoryURL]/index.sucatalog and not worry about making changes for every operating system:
1:  RewriteEngine On  
2:  Options FollowSymLinks  
3:  RewriteBase /  
4:  RewriteCond %{HTTP_USER_AGENT} Darwin/8  
5:  RewriteRule ^index(.*)\.sucatalog$ content/catalogs/index$1.sucatalog [L]  
6:  RewriteCond %{HTTP_USER_AGENT} Darwin/9  
7:  RewriteRule ^index(.*)\.sucatalog$ content/catalogs/others/index-leopard.merged-1$1.sucatalog [L]  
8:  RewriteCond %{HTTP_USER_AGENT} Darwin/10  
9:  RewriteRule ^index(.*)\.sucatalog$ content/catalogs/others/index-leopard-snowleopard.merged-1$1.sucatalog [L]  
10:  RewriteCond %{HTTP_USER_AGENT} Darwin/11  
11:  RewriteRule ^index(.*)\.sucatalog$ content/catalogs/others/index-lion-snowleopard-leopard.merged-1$1.sucatalog [L]  
12:  RewriteCond %{HTTP_USER_AGENT} Darwin/12  
13:  RewriteRule ^index(.*)\.sucatalog$ content/catalogs/others/index-mountainlion-lion-snowleopard-leopard.merged-1$1.sucatalog [L]  
14:  RewriteCond %{HTTP_USER_AGENT} Darwin/13  
15:  RewriteRule ^index(.*)\.sucatalog$ content/catalogs/others/index-10.9-mountainlion-lion-snowleopard-leopard.merged-1$1.sucatalog [L]  

Let’s start the web server and  check to make everything is OK:
sudo apachectl restart

Check the log files for errors:
tail /var/log/apache2/error_log
tail /var/log/apache2/access_log


After the repo_sync has finished. point one or more of your clients to the new repo and test, test, test:
sudo defaults write /Library/Preferences/com.apple.SoftwareUpdate CatalogURL “http://myupdateserver.com/index.sucatalog”

Verify that your entry was done properly:
defaults read /Library/Preferences/com.apple.SoftwareUpdate CatalogURL
http://myupdateserver.com/index.sucatalog

Test your client:
sudo softwareupdate -l

You should see the computer checking similar to below:

Software Update Tool
Copyright 2002-2010 Apple


Software Update found the following new or updated software:
   * SecUpdSrvr2014-003Lion-1.0
     Security Update 2014-003 (1.0), 180667K [recommended] [restart]
   * SecUpdSrvr2014-002Lion-1.0
     Security Update 2014-002 (1.0), 173813K [recommended] [restart]

If you see any errors, check your CatalogURL and try again.

At the server, check the log files to ensure that no errors have occurred. I like to go back 50 or 100 lines or so to make sure I’m seeing the beginning of the process:

tail -n50 /var/log/apache2/error_log
tail -n50 /var/log/apache2/access_log

Congratulations!! You now have a functioning update server!

Now let’s schedule it to sync nightly:

Create a plist file (reposync.plist) in the Reposado working directory with the following contents:

1:  <?xml version="1.0" encoding="UTF-8"?>  
2:  <!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">  
3:  <plist version="1.0">  
4:  <dict>  
5:    <key>Label</key>  
6:    <string>com.yourcompany.reposync</string>  
7:    <key>OnDemand</key>  
8:    <true/>  
9:    <key>RunAtLoad</key>  
10:    <false/>  
11:    <key>Program</key>  
12:    <string>/PATH/TO/SCRIPT</string>  
13:    <key>ProgramArguments</key>  
14:    <array>  
15:      <string>/usr/local/reposado/repo_sync</string>  
16:    </array>  
17:    <key>StartCalendarInterval</key>  
18:    <dict>  
19:      <key>Hour</key>  
20:      <integer>02</integer>  
21:      <key>Minute</key>  
22:      <integer>00</integer>  
23:    </dict>  
24:  </dict>  
25:  </plist>  

This plist will run repo_sync every night at 2:00 am. Copy it to /Library/LaunchDaemons/com.mycompany.reposync.plist:
cp reposync.plist /Library/LaunchDaemons/com.mycompany.reposync.plist

and launch it:
sudo launchctl load -w /Library/LaunchDaemons/com.mycompany.reposync.plist

Next steps:



Wednesday, July 16, 2014

Cleaning up the network monitoring

Last week I had the pleasure/honor of being able to attend the Penn State Mac Admins Conference. But, before i get to implementing what I learned there are a free things that I need to clean up.  One of the issues that I was having prior to leaving was figuring out why our servers weren't recovering properly from power outages. (Before anybody starts screaming "you should have a UPS!", we have them but you can only maintain power for so long). We tend to have seasonal power outages and brown outs at this time of year as well as people being helpful and unplugging the server racks. While things usually recover nicely, we've been having some timing issues. The servers are coming back up before the network. And since they don't have the network available, DNS and DHCP epithet don't or sort of start up. I need to come up d with a way of monitoring this with my current console. I've been using Opsview core for a couple of years to do general system monitoring. It's based on Nagios and had a semi useful web interface, so I can pretty much bend it to my will.

Post PSUMac ToDo List

Post Conference ToDo List

Last week I had the honor/pleasure of attending the MacAdministrator's Conference at Penn State University. It was a fun filled and brain filling week of techy goodness.  Below is the short version of the todo list that I came up with:

  1. Cleaning up and finishing some of my deployment scripts.
  2. Reposado (https://github.com/wdas/reposado) 
    1. Multi-Site alterations (http://seankaiser.com/blog/2013/05/23/multi-site-reposado/) 
  3. Autopkg (https://github.com/autopkg/autopkg) 
    1. JSS addon (https://github.com/arubdesu/jss-autopkg-addon) 
    2. Autopkg change notifications (http://seankaiser.com/blog/2013/12/16/autopkg-change-notifications/) 
  4. Git (https://about.gitlab.com/gitlab-ce/) 
    1. Workstation client (Sourcetree?) 
    2. Server solution (Git lab??)
  5. Changes to computer preference management.
    1. Change "defaults write" and plist packages to profiles (http://talkingmoose.net/2014/07/12/posting-my-psu-macadmins-conference-presentation/)