Page History

...

Adds a new dataset to the catalog

 Usage:

Wiki Markup&nbsp;&nbsp;&nbsp; datacat registerDataset \    datacat registerDataset [-options\] <data type> <logical folder> <file path> \\

Required Parameters:

  <data type>        Type of data in the file (merit, MC, DIGI, RECON, etc.)  See Java API child page for a full list.
  <logical folder>   Dataset Folder Path under which to create the new dataset.
  <file path>          Physical location of file to add to Data Catalog.

...

Adds an additional physical location to an existing dataset.  Use this routine to specify that a dataset exists in more than one physical location (ie:  it's on SLAC NFS and in SLAC XROOT.)  Except for <file path> all of the parameters and options are used to identify the existing dataset entry to which you want to add an additional physical location.

 Usage:

Wiki Markup&nbsp;&nbsp;&nbsp; datacat addLocation \    datacat addLocation [-options\] <dataset name> <logical folder> <file path>

 Required Parameters:

  <dataset name>     Name of existing dataset
  <logical folder>      Data Catalog Folder Path under which the dataset lives.
  <file path>             Additional physical location of file to add to the dataset entry.

...

Adds meta data entrie(s) to an existing dataset.

 Usage:

Wiki Markup&nbsp;&nbsp;&nbsp; datacat addMetaData \    datacat addMetaData [-options\] <logical folder>

 Required Parameters:
  <logical folder>   Logical Folder Path where the group or dataset lives, or to tag with meta data if no dataset or group specified.

...

As an example, dataset registration is performed by calling:

Wiki Markup&nbsp;   datacatalog.registerDataset(DATA_TYPE, DATA_CATALOG_LOCATION, DISK_LOCATION \ [, META_DATA\])

where:

• DATA_TYPE is the type of data within the file.
  • Typical values are MERIT, MC, RECON, ...
  • (See the Java API link below for a full list.)
  unmigrated-wiki-markup
• DATA_CATALOG_LOCATION has the following form:&nbsp;   <logical folder path>\[<dataset group name>:\]<dataset name>
  • <logical folder path> is required and has the form: /folder1/sub-folder/.../
    • It denotes the location within the Data Catalog folder-tree where the dataset will be registered.
    • The folder need not exist, it will be created if necessary.
  • <dataset group name> is optional.
    • If present, it must be followed by a ":" (colon) character.
    • The name is a simple alphanumeric string (spaces are not permitted.)
    • A dataset group is used to bundle together datasets which are fragments of a larger dataset.
    • For example, all merit files of a large monte carlo task are generally cataloged together using a dataset group.
    • <dataset name> is required.
      • It is simply the name of the dataset.
      • It is an alphanumeric string (spaces are not permitted.)
      • It must be unique within the folder or group where it will be placed.
  Wiki Markup
• DISK_LOCATION has the following form: <disk file path>\[@<site name>\]
  • <disk file path> is required.
    • It is the full path on disk (or in XRootd, etc.) to the file that is being registered.
  • <site name> is optional.
    • If specified, it must be preceded by a "@" (ampersand) character.
    • The site name tells the data catalog where to find the physical file.
    • Currently it may be one of:
      • SLAC, SLAC_XROOT, IN2P3, IN2P3_HPSS, UW
      • If no site name is specified, a default of "SLAC" is assumed.
  unmigrated-wiki-markup
• META_DATA is optional.&nbsp;   If specified, the supplied meta-data will be attached to the dataset upon registration.&nbsp;   Meta-data provide a basis for searching the Data Catalog for datasets.&nbsp;   A META_DATA expression has the following form: <name>=<value>\[:<name2>=<value2>\[...\]\]
  • <name> is required.
    • It is simply the name of the meta-data object, but it's form is significant because it denotes the object type of the <value> parameter. The Data Catalog will perform a type conversion and store the <value> parameter internally based on the type specified by the name:
      unmigrated-wiki-markup
      • n\[A-Z\]+.\* (ex: nEvents, nSecondsMET) indicates a numeric valueunmigrated-wiki-markup
      • t\[A-Z\]+.\* (ex: tStartDate, tEndDate) indicates a timestamp value
      • Anything else (ex: RunStatus, myDogsName) indicates a string value
  • <value> is required and must be separated from <name> by a single '=' (equals) character.
    • The value must reflect the type specified by <name> or an error will be thrown, and the registration will fail.
      • Numeric values have 38 decimal digits of precision for integers and 18 for floats. Leading and trailing zeros will be removed during conversion.unmigrated-wiki-markup
      • Timestamp values must be supplied in the following format: yyyy-mm-dd hh:mm:ss.\[fff...\]
        (fff... is an optional, fractional seconds component with nanosecond precision.)
      • String values are simply ASCII strings. Put whatever you want in there, even numbers.
  • Multiple <name>=<value> pairs may be supplied if separated by ":" (colon) characters
    

...