Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The

...

first completed implementation in the new refactorised scoreboarding code is a scheme for checkpointing and crash-recovery of the scoreboards and alignment of the scoreboarding with regular clock-hours/day-periods.

...

Both

...

of

...

these

...

improvements

...

are

...

discussed

...

below:

...

CheckPointing

In order to implement check-pointing,

...

I

...

save

...

the

...

state

...

of

...

the

...

scoreboards

...

to

...

disk,

...

while

...

processing

...

the

...

scoreboards

...

for

...

each

...

flow-file.

...

When

...

the

...

program

...

starts

...

for

...

the

...

first

...

time,

...

it

...

checks

...

for

...

checkpoints

...

on

...

disk

...

before

...

starting

...

the

...

scoreboading

...

of

...

every

...

direction

...

and

...

if

...

a

...

checkpoint

...

is

...

present

...

it

...

unmarshalls

...

the

...

data

...

from

...

the

...

checkpoint

...

and

...

starts

...

updating

...

the

...

scoreboards

...

from

...

that

...

point

...

on.

...

The

...

first

...

issue

...

was

...

how

...

to

...

save

...

the

...

state

...

to

...

disk.

...

I

...

decided

...

to

...

go

...

with

...

marshalling

...

and

...

saving

...

the

...

part

...

of

...

the

...

nested

...

hash

...

structure  containing the current scoreboard to disk. Specifically, mylist {direction}{$direction} {scoreboard} {aggregate} {report}

...

points

...

to

...

an

...

array

...

of

...

nested

...

hash

...

strcutures.

...


I

...

am

...

using

...

a

...

module

...

written

...