Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Image Added

Introduction

This is a presentation of the proposed adoption of an Enterprise-wide Service Oriented Architecture (SOA) for the GLAST experiment's data processing requirements. This project is composd of two parts, the GRITS Framework and the GRITS Projects that are built upon the framework. This presentation is primarily concerned with the GRITS Framework, which is a lightweight Java based distributed architecture which uses some, but not all, of the technologies from the J2EE specification.

The GRITS Framework is completely in place, and development of the various GRITS Projects within the framework has only recently begun. The first concrete components for the Processing Pipeline (which is one of the GRITS Projects) were recently installed into production.

The outline of this presentation is as follows:

  1. The GRITS Framework is introduced, which includes a description of the project's goals, the players involved, current project status and the project's timeline.
  2. The GRITS Projects are briefly introduced, with an emphasis on their requirements and use-cases. It is these requirements and use-cases that drove the design and implementaiton of the GRITS Framework.
  3. The technologies used by the GRITS Framework are described in some detail.
  4. Development tools and methodologies used by the GRITS Framework are described in some detail, as they are the recommended approach for development of GRITS Projects.

Anchor
framework
framework
GRITS Framework

The GRITS Framework is the infrastructure upon which GRITS Projects are intended to be devloped. The GRITS Framework was born out of a detailed deconstruction of the current, and proposed, GRITS Projects by members of the GLAST Core Team in early 2004. This deconstruction and subsequent analysis identified many common services shared by all GRITS Projects, and an SOA approach to deliver these sercies was unanamously agreed upon.

Anchor
projects
projects
GRITS Projects

The GLAST Projects are intended to serve the data processing needs of the GLAST collaboration. The current and proposed set of projects, and the platforms they are required to run on, are shown in the following table:

Project

Linux

Windows

Processing Pipeline (GINO)

(tick)

 

Release Manager

(tick)

(tick)

Installer

(tick)

(tick)

Tag Collector

(tick)

 

System Tests

(tick)

(tick)

Data Access

(tick)

(tick)

Anchor
common_services
common_services
GRITS Common Services

The GRITS Projects share a common set of services, which are listed in the following table.

Requirement

GINO

RM

Installer

TC

System Tests

Data Access

Web configuration/editing

(tick)

 

 

 

(tick)

 

Web reports

(tick)

(tick)

(tick)

(tick)

(tick)

(tick)

Single Sign-On

(tick)

(tick)

(tick)

(tick)

(tick)

(tick)

Persistence (CRUD)

(tick)

 

 

 

(tick)

 

Reports (DB Queries)

(tick)

(tick)

(tick)

(tick)

(tick)

(tick)

Batch Submission

(tick)

(tick)

 

 

(tick)

 

Scheduler

(tick)

(tick)

 

(tick)

(tick)

 

Event Notification

(tick)

(tick)

 

(tick)

(tick)

 

Automated email

(tick)

(tick)

 

 

 

(tick)

The bulk of these services are provided out-of-the-box by the GRITS Framework technologies. Those that aren't are provided by the grits commons project.

A shared set of common services was one of the primary reasons for adopting an SOA.

Anchor
legacy_services
legacy_services
Legacy Systems and Services

Another important reason for adopting an SOA is to leverage and integrate existing back-end systems services. This set of legacy systems and services and the platforms they run on are shown in the following table.

Legacy System

Linux

Windows

Oracle

(tick)

(tick)

LSF

(tick)

(tick)

NFS

(tick)

(tick)

Windows File System

 

(tick)

mstore

(tick)

 

cvs

(tick)

(tick)

gcc

(tick)

 

cl.exe

 

(tick)

cmt

(tick)

(tick)

gleam.exe

(tick)

(tick)

ROOT

(tick)

(tick)

Anchor
goals
goals
GRITS Framework Goals

The primary goal of the GRITS Framework is to provide the software architecture and infrastructure for the GLAST Core Team to accomplish its mission. Specific goals include:

  • Provide a common framework to develop the GRITS Projects and their associated set of common services
  • Provide a distributed, cross-platform SOA for GLAST Projects that is simple to develop with and simple to use
  • The framework and technologies must have proven to be lightweight, scaleable maintainable, and enjoy wide adoption in the community
  • Leverage existing SLAC infrastructure
  • Promote team oriented development (as opposed to individual contributors)
  • Maximize disparate talents of a small group
    • Programmers
    • Web Developers
    • Occasionally-connected programmer/manager/astronomer
  • Utilize talents of JAS group

There are three sets of expected users of the GLAST Framework and GLAST Projects and their timelines.

User

Description

Time Scale

Core team

Develop and maintain GLAST data processing projects

now

GLAST collaborators

Use the GLAST data processing projects to accomplish the GLAST mission

now

Astronomy community at large

Consumers of GLAST data products

post-launch (2006)

Anchor
technologies
technologies
GRITS Framework Technologies

The GRITS Framework primarily uses open-source software, but also containing commercial components where they make sense.

The primary components are:

    • The Spring Framework, which is the cornerstone of the GRITS Framework, provides those parts of the J2EE specification that we rejected as either too complicated, too expensive, or otherwise not addressing the project's requirements.
    • Hibernate, a thin wrapper over JDBC which provides the full power of transparent persistence by providing O/R mapping but not trying to hide the underlying relational database.
    • Web servers (IIS on Windows and Apache on Linux) for security-conscious service deployment.
    • ColdFusion MX for web front-end development.
    • Several Jakarta components, primarily Tomcat for service deployment and many of the jakarta commons for

An Example Hibernate Mapping File

Code Block
xml
xml
Wiki Markup

!J2EE for GLAST.png|align=center!

h2. Introduction

This is a presentation of the proposed adoption of an Enterprise-wide Service Oriented Architecture (SOA) for the GLAST experiment's data processing requirements. This project is composd of two parts, the [GRITS Framework|framework] and the [GRITS Projects|projects] that are built upon the framework. This presentation is primarily concerned with the _GRITS Framework_, which is a lightweight Java based distributed architecture which uses some, but not all, of the technologies from the J2EE specification.

The _GRITS Framework_ is completely in place, and development of the various _GRITS Projects_ within the framework has only recently begun. The first concrete components for the *Processing Pipeline* (which is one of the _GRITS Projects_) were recently installed into production.

The outline of this presentation is as follows:

# The _GRITS Framework_ is introduced, which includes a description of the project's goals, the players involved, current project status and the project's timeline.
# The _GRITS Projects_ are briefly introduced, with an emphasis on their requirements and use-cases. It is these requirements and use-cases that drove the design and implementaiton of the _GRITS Framework_.
# The technologies used by the _GRITS Framework_ are described in some detail.
# Development tools and methodologies used by the _GRITS Framework_ are described in some detail, as they are the recommended approach for development of _GRITS Projects_.

h2. {anchor:framework}GRITS Framework

The _GRITS Framework_ is the infrastructure upon which _GRITS Projects_ are intended to be devloped. The _GRITS Framework_ was born out of a detailed deconstruction of the current, and proposed, _GRITS Projects_ by members of the GLAST Core Team in early 2004. This deconstruction and subsequent analysis identified many [common services|services] shared by all _GRITS Projects_, and an SOA approach to deliver these sercies was unanamously agreed upon.

h3. {anchor:projects}GRITS Projects

The GLAST Projects are intended to serve the data processing needs of the GLAST collaboration. The current and proposed set of projects,  and the platforms they are required to run on, are shown in the following table:

||Project||Linux||Windows||
|Processing Pipeline (GINO)|(/)| |
|Release Manager|(/)|(/)|
|Installer|(/)|(/)|
|Tag Collector|(/)| |
|System Tests|(/)|(/)|
|*Data Access*|(/)|(/)|

h3. {anchor:common_services} GRITS Common Services

The _GRITS Projects_ share a common set of services, which are listed in the following table. 

||Requirement||GINO||RM||Installer||TC||System Tests||Data Access||
|Web configuration/editing|(/)| | | | (/)| |
|Web reports|(/)|(/)|(/)|(/)|(/)|(/)|
|Single Sign-On|(/)|(/)|(/)|(/)|(/)|(/)|
|Persistence (CRUD)|(/)| | | |(/)| |
|Reports (DB Queries)|(/)|(/)|(/)|(/)|(/)|(/)|
|Batch Submission|(/)|(/)| | |(/)| |
|Scheduler|(/)|(/)| |(/)|(/)| |
|Event Notification|(/)|(/)| |(/)|(/)| |
|Automated email|(/)|(/)| | | |(/)|

The bulk of these services are provided out-of-the-box by the _GRITS Framework_ technologies. Those that aren't are provided by the _grits commons_ project.

{quote}
A shared set of common services was one of the primary reasons for adopting an SOA.
{quote}

h3. {anchor:legacy_services} Legacy Systems and Services

Another important reason for adopting an SOA is to leverage and integrate existing back-end systems services. This set of _legacy_ systems and services and the platforms they run on are shown in the following table.

||Legacy System||Linux||Windows||
|Oracle|(/)|(/)|
|LSF|(/)|(/)|
|NFS|(/)|(/)|
|Windows File System| |(/)|
|mstore|(/)| |
|cvs|(/)|(/)|
|gcc|(/)| |
|cl.exe| |(/)|
|cmt|(/)|(/)|
|gleam.exe|(/)|(/)|
|ROOT|(/)|(/)|

h3. {anchor|goals} GRITS Framework Goals

The primary goal of the _GRITS Framework_ is to provide the software architecture and infrastructure for the GLAST Core Team to accomplish its mission. Specific goals include:

- Provide a common framework to develop the [GRITS Projects|projects] and their associated set of [common services|common_services]
- Provide a distributed, cross-platform SOA for _GLAST Projects_ that is simple to develop with and simple to use
- The framework and technologies must have proven to be lightweight, scaleable maintainable, and enjoy wide adoption in the community
- Leverage existing SLAC infrastructure
- Promote team oriented development (as opposed to individual contributors)
- Maximize disparate talents of a small group
-- Programmers
-- Web Developers
-- Occasionally-connected programmer/manager/astronomer
- Utilize talents of JAS group

There are three sets of expected users of the _GLAST Framework_ and _GLAST Projects_ and their timelines.

||User||Description||Time Scale||
|Core team|Develop and maintain  [GLAST data processing projects|#projects]|now|
|GLAST collaborators|Use the  [GLAST data processing projects|#projects] to accomplish the GLAST mission|now|
|Astronomy community at large|Consumers of GLAST data products|post-launch (2006)|


h3. {anchor|technologies}GRITS Framework Technologies

The _GRITS Framework_ primarily uses open-source software, but also containing commercial components where they make sense.

 The primary components are:
#- The [Spring Framework|http://www.springframework.org/], which is the cornerstone of the _GRITS Framework_, provides those parts of the J2EE specification that we rejected as either too complicated, too expensive, or otherwise not addressing the project's requirements.
#- [Hibernate|http://www.hibernate.org/], a thin wrapper over JDBC which provides the full power of transparent persistence by providing O/R mapping but not trying to hide the underlying relational database.
#- Web servers ([IIS|http://www.microsoft.com/WindowsServer2003/iis/default.mspx] on Windows and [Apache|http://httpd.apache.org/] on Linux) for security-conscious service deployment.
#- [ColdFusion MX|http://www.macromedia.com/devnet/mx/coldfusion/] for web front-end development.
#- Several [Jakarta|http://jakarta.apache.org/] components, primarily [Tomcat|http://jakarta.apache.org/tomcat/] for service deployment  and many of the [jakarta commons|http://jakarta.apache.org/commons/] for 



h2. An Example Hibernate Mapping File

{code:xml}
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE hibernate-mapping PUBLIC
        "-//Hibernate/Hibernate Mapping DTD//EN"
        "http://hibernate.sourceforge.net/hibernate-mapping-2.0.dtd">

<hibernate-mapping package="grits.gino.domain">

	<class name="TaskImpl" table="TASK">

		<id name="id" type="long" unsaved-value="null">
			<column name="TASK_PK" />
			<generator class="native">
				<param name="sequence">TASK_SEQ</param>
			</generator>
		</id>

		<property name="name" type="string">
			<column name="TASKNAME" length="30" unique="true"
				not-null="true" />
		</property>

		<property name="datasetBasePath" type="string">
			<column name="BASEFILEPATH" length="200" not-null="false" />
		</property>

		<property name="runLogPath" type="string">
			<column name="RUNLOGFILEPATH" length="200" not-null="true" />
		</property>

		<many-to-one name="type">
			<column name="TASKTYPE_FK" not-null="true" />
		</many-to-one>

		<set name="taskProcessSortedSet" inverse="true"
			cascade="all-delete-orphan" sort="natural" order-by="SEQUENCE asc">

			<key column="TASK_FK" />
			<one-to-many class="TaskProcessImpl" />
		</set>

		<set name="datasetSetInternal" inverse="true"
			cascade="all-delete-orphan">

			<key column="TASK_FK" />
			<one-to-many class="DatasetImpl" />
		</set>

	</class>

</hibernate-mapping>
{code}