Defining a release. ------------------ A release of the CDF Run2 software is defined by the .newver and .newrel files in the Release/Releases area. There are other .* files in this area which serve as documentation and historical logs. The .newver file is a shell script. The .newrel file is an input to the newrel utility. Both of these files, as well as the .doc file are generated initially by the Changed_packages script and modified by the retagger script. Another script, also created by the Changed_packages script, is the .tags file. This is the script executed by the Tag_release script. It is NOT updated by the retagger because the retagger does its tagging dynamically. The .newver script is used by the Prebuild_new_release script to retrieve the necessary tagged versions of packages to disk in preparation for building a release. The .newrel file is used by the newrel utility also executed by the Prebuild_new_release script to create the release directory structure and the necessary links. ----------------------------------- How to "Cut" a Release. NOTE: All scripts (not . specific) live in the Scripts directory of the Release package (even in development). Each higher level "knows" about the next lower level (the immediate preceeding activity) and is capable of executing it with "appropriate" defaults. If ALL defaults are to be taken, it is possible to "cut" a release with a single command ASSUMING ALL REPOSITORY PERMISSIONS ARE CORRECT. Tag the changed packages. The tag operation is done ONCE on ONE machine because it ONLY changes the repositories. Run the Changed_packages script. Run Changed_packages script to determine which packages have changed, what the new tag values should be, and generate the . files which provide the information necessary to complete the tagging and building activity. Adds the . files to the Release package Releases directory in the cdfcvs (CDF) repository. Parameters: Reference release: The release from which changes are to be noted. release name: The string which names the release. Tag date: OPTIONAL - default the value returned by make_midnight script. The date of the last revision to be tagged. Input files: None. Output files: stdout .doc File for the announcement detailing which packages and tags were included in the release and the number of changes in each. .newver Script to bring the package versions to disk using newver utility. .newrel File, used by newrel utility, to create the release directory and the package links. .tags Script to actually perform the tag action in the cvs repositories. Permissions required: Write permission to Release package in cdfcvs (CDF) repository. Run Tag_release script. Basically, the Tag_release script only retrieves the Release package from the repository (to access the latest . files) and runs the .tags script adding the tag log to the Release/Releases repository area. However, if the Changed_packages script has not been run, the .tags script will not exist. Tag release will, in this case, run the Changed packages script with a reference release of the preceeding frozen release (3.6.x will use the date of 3.5.0 as the reference release). This is least likely to miss any updates without being overly redundant. Parameters: release, as above. Input files: None. Output files: stdout .tagged The log (stdout) for the .tags execution. Permissions required: Write permission to add the tag log (.tagged) to the Release package and update the cdfcvs (CDF) repository. ------------------------------------------------------------------------ NOTE: For the following activities your login id MUST be cdfsoft so that you can write in that area and the ownership of the files will be assigned properly. These activities are performed on EACH machine for which a build is required. Bring the package versions to local disk. Run the Prebuild_new_release script. At the most basic level, the Prebuild_new_release script performs two functions, it runs the .newver script to bring the packages to disk and runs newrel with the .newrel file. However, in detail: 1. Checks out the latest copy of the Release package and determines if .tagged file exists. If it does not, Prebuild_new_release runs the Tag_release script after prompting for a userid@node under which to tag the release (its nice to be able to look in the repository history and determine which human tagged a release). 2. Provides output sufficient to determine if sufficient disk space exists to contain a build of this release and prompts for permission to continue. 3. Determines if Visualization is to be built for this machine. If it is not to be built here, the references are removed from the .newrel and .newver files. 4. Runs the .newver script. Creating newver log. Please note: This script attempts to bring ALL required package versions to disk, whether retagged or not. Errors are reported if the package version is already on the disk. However, it avoids the problem of attempting to build on machines which do not have up to date releases. 5. If the directory does not exist, runs the newrel utility with .newrel. Else, if the .newlinks file (generated by retagger script) exists, runs rel_link script to update links. Parameters: release string release tag: OPTIONAL defaults to the latest version of the Release package on the trunk. This could be useful for building retagged (usually only intermediate releases) of packages when the latest retag isn't desired. CAUTION This is only added because its possible. Input files: stdin .newrel From Changed_packages or retagger script (repository) .newver From Changed_packages or retagger script (repository) Output files: stdout .newver.log stdout from .newver script execution. Permissions required: Write permission in $BFDIST. Usually $BFDIST is owned by cdfsoft. Read access to the cvs repositories. Build the release. Run the Build_release script. The Build_release script in simplest term does a gmake in the area under $BFDIST. Functionality: 1. Determines if the Prebuild_new_release scripts has been run. (IF the exists Prebuild_new_release is assumed to have done it.) If not runs Prebuild_new_release. 2. Determines if there have been updates since the last Prebuild_new_release execution. (Checks out Releases package. compares dates on .newver and .newver.log.) If so, prompts for permission to run Prebuild_new_release. If granted, runs Prebuild_new_release. 3. Declares and configures . Sets up and cd's to release area. 4. If Visualization is to be built, sets up Visualization environment. 5. If clean build requested, issues gmake clean. 6. Uses the rebuild scriptto actually perform the gmake activity. 7. Sends mail to cdf_code_management@fnal.gov with gmake (error) messages from gmake.log. Parameters: release build type OPTIONAL default "" currently only "clean" is meaningful. Alternate form: b[uild_type]=clean Q{ual]=srt_qual Specify the srt_qual value. default: Q=default p[arallel]=parallel Specify the degree of parallelization (number of concurrent processes) for the build. Example: p=8 Default is no parallelization. Input files: stdin .newver.log from Prebuild_new_release Output files: stdout $BFDIST/releases//results/gmake.log email to cdfcode_management@fnal.gov Permissions required: Same as Prebuild_new_release. --------------------------------- Retagging the release. At some point in the Tag/Build cycle toward a frozen release it is usually necessary to retag packages in the repository either because a bug has been fixed in a newer revision, or additional functionality has been added, or, in some cases, an OLDER revision is considered to be a better fit with other packages. Whatever the reason, there is a script to provide this capability. Running the retagger script. The retagger script reads and parses the file determining the package and files to be retagged For each package, it extracts the current tag from the .newrel file. (Note that it is assumed that the release has been tagged before being retagged.) A new tag value is generated in the same manner as for Changed_packages. If individual files are to be retagged, the package as tagged with the current tag is retagged with the new tag. The individual files are then "detagged" and the revisions requested retagged with the new tag. This provides a trivial backstep to the current. If individual files are not to be retagged, the package as of a date, or tag is then retagged. The . files are updated to indicate the new status and a .newlinks file is added to the repository for use in the Prebuild_new_release script. Parameters: release files_2_retag - OPTIONAL default "files_2_retag.txt" name of the file containin package/file information. input files: files_2_retag file output files: stdout .newlinks Permissions required: Write access to any packages to be retagged and Release package. Format of file ------------------------------ package/filespec package/filespec2 package1/filespec3 package3 package4 ... ------------------------------------ NOTE: Any line beginning with a '#' is a comment. Entries for a single package are single spaced (No blank lines) Each PACKAGE ends with a single blank(empty) line - including the last. This is used by the retagger and is a visual aid in grouping the file retags by package. The each line contains single spaced entries beginning in the first space. If no revision/date/tag information is included on a line, the tag will be applied to the head. ---------------------------------------- Cloning a release. ----------------- Often, it is desirable to "base" a release on an earlier release with a "few" changes. One first must clone, (copy the release definition) the release. The clone can then be be modified by retagging the elements which contain the changes. Clone_release script Parameters: Name of existing release Name to be given to the new release Revision: OPTIONAL - used same as Prebuild_new_release. Applied to existing release. Input files: Existing release definition files from the Release package in the repository. Output files: New release definition files for the new release (cloned from the existing release) in the repository. Permissions required: Write access to the Release package in the repository. Debugging --------- It is possible due to the vagarities of CVS to miss tagging some files in packages. The Check_tags script can reduce the search for these by detecting those files which "should" have been tagged (because the latest revision on the trunk was "live" at the effective tag time). It writes an abbreviated log on the console and a detailed log in an announced area. This script was used to change the algorithm in changed_packages which became Changed_packages as a result. It was determined that the older method of setting the date for the entire changed_packages was prone to error. A more complex algorithm based on the reference tag on each package was implemented. Arguments: The release to be checked. Output: stdout detailed log. Permissions: Abilility to read the cvs repositories. Utility scripts --------------- The following scripts are used by the above as needed: Find_cvs_root: Determines the read and read/write CVSROOT values and the repository for a package. Find_package_changes: determines the number of changes to a package. Get_last_tag_from_file: Determines the last tag value applied to to a package. Make_midnight: creates the cvs acceptable time definition for one minute after midnight this morning. This is an actual date/time string. define_work_area: determines the appropriate directory to use for temporary work area. make_new_tag: Increments the "standard" tag. rel_link: fixes the links for retagged packages in an existing release.