This document is meant as a manual for administering the OpenMadrigal web site. It contains information on both the developer's site www.openmadrigal.org and the archive site cedar.openmadrigal.org.
The cedar.openmadrigal.org site serves as a central archive of all Madrigal sites. It also serves as a home for instrument data not maintained by local Madrigal sites. It is a standard Madrigal site, except that it uses a modified version of the homepage index.html. The modified version of index may be found in CVS under MillstoneCVS/Madrigal/OpenMadrigal/CEDAR_site.
All data is downloaded from remote sites using the script import_madrigal.py. This script may be found in CVS under MillstoneCVS/Madrigal/OpenMadrigal/scripts. It should be set up as a cron job to download remote Madrigal data on a regular basis, according to agreements with remote Madrigal site administrators. All data downloaded is logged in /opt/cedar/metadata/userdata/<site>_import_<year>.txt.
The script changeInstAccess.py can be useful in administering this site. It allows you to change the status of all experiments with a given instrument kinst with one command. This script may be found in CVS under MillstoneCVS/Madrigal/OpenMadrigal/scripts.
The OpenMadrigal developer's web site serves a number of purposes:
This manual also covers the building and testing of a new Madrigal release. All files relating to the OpenMadrigal.org website (including this document) are located in CVS under Millstone/Madrigal/OpenMadrigal.
Two cron jobs are required to collect these data, createExpUpdate.py and createGeoUpdate.py.
createExpUpdate.py - located in CVS under Millstone/Madrigal/OpenMadrigal/scripts. Usage: createExpUpdate.py [-e emailAddress] [-e emailAddress2] ... where any number of email addresses can be notified of the results. This script collects the metadata files expTab.txt and fileTab.txt from each of the Madrigal sites listed in siteTab.txt, and stores them in the directory metadata/[site_name]_[site_id]. The site_name and site_id are also set by siteTab.txt. This cron job should be set to run at least once a day, and the results should be emailed to the OpenMadrigal administrator. The results are accessed via the web from the url http://www.haystack.mit.edu/madrigal/distributionFiles/metadata.
createGeoUpdate.py - located in CVS under Millstone/Madrigal/OpenMadrigal/scripts. Usage: createGeoUpdate.py [-e emailAddress] [-e emailAddress2] ... where any number of email addresses can be notified of the results.. This script will update the geofil.tar.Z and ig_rz.dat file on OpenMadrigal. Also updates the file status.dat which indicates the last update date of each type of geophysical data. All the files updated are accessed via the url http://www.haystack.mit.edu/madrigal/distributionFiles. Due to relatively infrequent updates to the base files, this cron job is now run only once a week.
Three core metadata files are also distributed via updateMaster:
The OpenMadrigal administrator is responsible for manually updating these three files on http://www.haystack.mit.edu/madrigal/distributionFiles/metadata whenever they are changed. You can set up up the script checkMetadataDist.py located in CVS under Millstone/Madrigal/OpenMadrigal/scripts as a cron job to check that these files are in sync.
The script checkMetadata.py can be set up as a cron job to monitor whether all Madrigal sites have these three up-to-date metadata files. The sites need to be running Madrigal 2.5 or greater in order to have automatic update of these files. This script is located in Subversion under OpenMadrigalSVN/trunk/madroot/source/madpy/scripts/bin.
Finally, this capability requires that the cgi script compareToArchive.py be installed on the Millstone server at url http://madrigal.haystack.mit.edu/cgi-bin/madrigal/compareToArchive.py. This script can be found in Subversion under OpenMadrigalSVN/madroot/scripts/madpy/scripts/cgi.
The Madrigal derivation engine contains a number of models that are under active development, and so should be updated if possible with each new Madrigal release. These models are listed below.
All Fortran code that uses reals is converted to use all doubles using nag_apt tools.
From Subversion at Millstone, checkout OpenMadrigalSVN/trunk/madroot. The file source/configure.ac has the line AC_INIT([Madrigal] that should be updated with the Madrigal version. Since autotools is used in building a distribution, you want to be sure that your machine has up-to-date autotools. For Madrigal 2.5, I used autoconf 2.6.3, automake 1.10.2, libtool 2.2.4, and m4 1.4.12. To create the needed autotool files, cd to source and run:
To make the distribution file, cd to madroot, and run "make -f Makefile.gnu". The file MANIFEST in the madroot directory controls which files are included in the tar file. The general test procedure is to build the distribution, and then test install on any available unix machines, both as updates and as new installs. There are also regression tests to be run on the core C and Fortran libraries, and on the Python library.
The release number is set in source/configure.ac and in two places in madContents.html (title and top H1 line).
A regression test for the C and Fortrans libraries is runMadrecDiagnostics, found in Subversion under OpenMadrigalSVN/trunk/madroot/source/madc/madrec. This regression test can also be run under purify. To do so build the Fortran library geolib with a debug flag. Then build the madrec library using the Makefile.purify file in the madrec directory.
A second regression test for the geolib Fortran library is to run madf/geolib/testGeolib. It will produce a file testGeolib.out, which should diff with testGeolib.out.rock.
The python regression tests are in OpenMadrigal/madroot/source/madpy/scripts/regressionTests. Each of the following three regression tests are run as $MADROOT/bin/python script:
To rebuilding all the Madrigal documentation into a single pdf file, cd $MADROOT/doc and run
htmldoc --batch madDocBook.book
If you have added or removed any documentation files, or changed their order, you will need to use the htmldoc UI to make those changes. Also, in order to import the graphics into the generated pdf, you will need to make a temporary copy of the $MADROOT/icons directory in the $MADROOT/doc directory.