Dependency perl code

Special variables for the DPC

Defining type and the time variables

Any file which has the same filename as another file except for a change of date is assumed to be of the same type. So it would be assumed that /tmp/raw2009-12-02.dat (from 2 December 2009) was of the same type as /tmp/raw2011-05-12.dat (from 12 May 2011). And for the purposes of the DPC the type would be written as /tmp/rawYEAR-MONTH-MDAY.dat, where YEAR, MONTH and MDAY are time variables that represent the year, month and day of month respectively.

An example of code using these time variables is shown below

#--------------------------------------------
# Define the directories
#--------------------------------------------
Define;homeDir;/home/data;
#--------------------------------------------
# Move files into home directory
#--------------------------------------------
$homeDir$/rawYEAR-MONTH-MDAY.dat;/tmp/rawYEAR-MONTH-MDAY.dat;mv 
/tmp/rawYEAR-MONTH-MDAY.dat $homeDir$/rawYEAR-MONTH-MDAY.dat;

where the last two lines should be on one line. It shows the /tmp/rawYEAR-MONTH-MDAY.dat type of files are parents to the /home/data/rawYEAR-MONTH-MDAY.dat type of files. If the DPC is asked to update $homeDir$/rawYEAR-MONTH-MDAY.dat, it will find all its parent files. In this case, this is just a case of listing all the file of type /tmp/rawYEAR-MONTH-MDAY.dat, which would be done by carrying out a list on the string

/tmp/raw[12][09]??-[0-1][0-9]-[0-3][0-9].dat

which is fine provided the files are between the years 1900 and 2099. For each parent file, the DPC can determine the value of YEAR, MONTH and MDAY from the filename, and determine if its child of type /home/data/rawYEAR-MONTH-MDAY.dat needs updating.

In these circumstances, an advantage of this method is that the DPC doesn't need any previous knowledge of the dates expected to update the child files. It doesn't anticipate when files should be received, and so it doesn't matter to the DPC if source files arrive on time or six months late. It will just update all the file once the source files have arrived.

They are other rarer circumstances when the DPC cannot determine a date or a range of dates. This is true in the problems file, which is discussed in a later page, when specifying YEAR in a filename. In this case, there is no parent file to extract a date, so the problems file needs a starting year and end year to loop through. This is set in the set_environment.pl file (HREF!!). For MONTH, the DPC can just loop through 01 to 12 and for MDAY it can loop through 01 to 31 so these are easier to handle.

The time variables, like MONTH and MDAY, are similar to the loop variables described on the Define variables page. It would be reasonably easy to create a loop over the 12 months of the year, but to create a loop over the days of a month is significantly harder because the length of a month varies. For example, the user may define a montly products to require a file from each day of that month. In this case, it's necessary to know the length of the month (it's not possible to just loop from the 01 to 31, because 31 may never arrive). This is another reason for using the time variables, rather than the user defining their own loop variable.

As well as MDAY, MONTH and YEAR the other time variables are: YeaR, like YEAR put only the last two digits of YEAR; and DEKAD which is a complete dekad.

OBJECT and DEPEND

The other special variables are OBJECT and DEPEND. They are only used in the command part of the instruction line, and OBJECT is just short hand for the filename of the object and DEPEND is short hand for the list of parent files. So the example on the Dependency file page could be written

#--------------------------------------------
# Move data to home directory
#--------------------------------------------
/home/data/raw.dat;/tmp/raw.dat;mv DEPEND OBJECT;

where DEPEND will be replaced /tmp/raw.dat and OBJECT will be replaced by /home/data/raw.dat. If there is more than one parent file, DEPEND will be replaced by the list of parent files separated by a space.

All the special variables

The special variables are

DEKAD	This can appear in any component of an instruction line and is replaced with the dekad of the month as either 1, 2 or 3.
DEPEND	This string can appear in the command part of an instruction line and will be replaced by the list of parent files. Commas found in the parent string are replaced with spaces.
MDAY	This can appear in any component of an instruction line and is replaced with the day of the month as a two digit integer, e.g. 4th of month is written as 04.
MONTH	This can appear in any component of an instruction line and is replaced with the month as a two digit integer, e.g. May is written as 05.
OBJECT	This string can appear in the command part of an instruction line and will be replaced by the object file
YEAR	This can appear in any component of an instruction line and is replaced with the year as a four digit integer, e.g. 2009
YeaR	Like YEAR but YeaR is replaced with the last two digits of the year, e.g. for 2009 Year is replaced with 09.
[* ... *]	The name of a parent file can be followed by some auxiliary information, which is sandwiched by the characters `[*' and '*]'. More information on this can be seen on the auxiliary information page.