Page History
...
Backup
...
of
...
Xrootd
...
FGST
...
data
...
to
...
HPSS
...
The
...
FGST
...
xrootd
...
cluster
...
is
...
integrated with
...
HPSS.
...
New
...
files
...
that
...
are
...
copied
...
into
...
the
...
xrootd
...
cluster
...
are
...
marked
...
that
...
they
...
have
...
not
...
been
...
migrated
...
to
...
HPSS.
...
Not
...
all
...
files
...
that
...
are
...
copied
...
to
...
xrootd
...
are
...
backed
...
up
...
but
...
currently
...
only
...
some
...
of
...
the
...
L1
...
produced
...
files.
...
The
...
list
...
of
...
files
...
that
...
should
...
be
...
migrated
...
to
...
HPSS
...
is
...
produced
...
daily
...
and
...
is
...
used
...
by
...
the
...
migration
...
tools
...
to
...
transfer
...
files
...
to
...
HPSS.
...
Once
...
a
...
file
...
has
...
been
...
migrated
...
to
...
HPSS
...
it
...
is
...
marked
...
as
...
migrated
...
so
...
that
...
it
...
will
...
not
...
be
...
migrated
...
again.
...
There
...
are
...
three
...
lists
...
of
...
L1
...
files
...
to
...
backup,
...
found
...
in
...
~glast/_backupfiles
...
. Each line of these files contains:
size, checksum, filename
file name | abrev | use for Migration | comment |
|---|---|---|---|
level1backup-completefiles.txt |
...
min | No | All Level1 files | |
level1backup-fullfiles.txt | full | YES | subset of the previous list |
level1backupfiles.txt | complete | No | subset of the previous list |
| Composition Setup |
|---|
| YES | subset of the previous list | | level1backupfiles.txt | No| subset of the previous list | {composition-setup} deck.tab.active.border=3px solid blue {composition-setup} Currently the total number of files and accumulated file size in these three list are: A detail of all the file types, size and file count is shown {toggle-cloak:id=one} Show the detail of all the file types, size and file count is shown {cloak:id=one} |
The total number of files and accumulated file size in these three lists are:
| Wiki Markup |
|---|
{html-include:url=http://www.slac.stanford.edu/~wilko/glastmon/backuplist.html} |
Currently all files (including recon) from the full-listing are backed up to HPSS.
| Toggle Cloak | ||
|---|---|---|
|
Show the detail of all the file types, sizes in the three lists
| Cloak | ||
|---|---|---|
| ||
| Wiki Markup |
|---|
{html-include:url=http://www.slac.stanford.edu/~wilko/glastmon/l1backup.html}
{cloak}
The current HPSS tape usage for data backedup from the xrootd cluster to HPSS is:
{html-include:url=http://www.slac.stanford.edu/~wilko/glastmon/lastffam.html}
{toggle-cloak:id=one} Show table with sizes of components to backup
{cloak:id=one}
|
| Cloak |
|---|
The current HPSS tape usage for data backed up from the xrootd cluster to HPSS is:
| Wiki Markup |
|---|
{{html-include:url=http://www.slac.stanford.edu/~wilko/glastmon/l1backuplastffam.html} {cloak} h2. What is backed up The file list level1backup-fullfiles.txt is used but recon files are removed from this list. Therefore the components bq. digi filteredMerit ft1 ft2 ft2Seconds gcr ls1 merit recon svac are backed up. However there are some recon files in HPSS because a few month ago recon files were backed up. h2. Tape usage (20090918) The old HPSS contains about 53K files and 122 TB on 644 tapes. || Name | Files | Size in GB || | recon | 1843 | 51102.50 | | svac | 6675 | 33621.71 | | digi | 6676 | 29044.85 | | merit | 6676 | 10791.51 | | gcr | 6676 | 335.37 | | ls1 | 6675 | 138.04 | | filteredMerit | 428 | 17.59 | | ft1 | 6675 | 14.50 | | ft2Seconds | 5180 | 3.00 | | ft2 | 5539 | 0.18 | The new HPSS contains 8TB but these files are also in the old HPSS. |