Simulations Archives and Digital Simulation Libraries 

The analysis and visualization of data from large-scale simulations can be a 
complex task.  Simulation results are rarely reused, in part because users forget 
where data is located or how it was generated. Development of simulation archives 
and digital scientific libraries providing innovative techniques for automating the 
processes of data indexing and archiving can alleviate this problem. Such a system 
will automatically log the output of a simulation along with a description of the 
problem, code, version, and parameter values into an online index.  A web-based 
front end can now facilitate access to this information and data. Each interaction 
server in the proposed architecture has a service handler providing a JDBC interface 
to a local or remote database. This will enable transparent and uniform access to 
different database and archival systems (e.g. Active Data Repository) for storage and 
retrieval of simulations, experimental and observational data. It can additionally be 
used to maintain checkpoint information for large simulations. We hope that this 
organized access to simulation results will eliminate redundant runs and increase the 
range of analyses performed by users leading to more effective collaborations. 

Enabling Collaboration 

The objective of the collaboration support is to enable distributed and heterogeneous 
(wired and wireless) clients to collaborate and share knowledge while interacting with, 
analyzing and controlling simulations.  Our goal is to support dynamic collaboration 
environment number of collaborating clients, their locations, capabilities and 
interests are dynamic.  
 

Schematic of Server - Client Interaction in the prototype implementation of DISCOVER

How does DISCOVER work ?

Given the mechanisms required to achieve this web-enabled collaborative computational portal, here is a short description of their implementation in DISCOVER:

Important Classes in DISCOVER include:
At the Client

• Client: Where every GUI action is handled retrieving a response from the application
• ClientResponseThread: Polling for responses

At the Server

• MainThread: Initializes the connection with the application, passes commands to the application and handles Global updates
• ResponseThread: Builds response Objects ready to be picked up by the clients in CollaborationHandler
• CollaborationHandler: Handles the GET requests from clients to pick up Responses and Chat and WhiteBoard InfoPackets through POST requests
• CommandHandler: Where the commands are issued from the client through a POST request. Prepares the view/command request to be sent to the application
• LoginHandler: Authenticates users
• MasterServlet: Sends Jar file to client

 

Mechanism	Client	MiddleWare	Backend
i - Global Updates	A polling mechanism inClientResponseThreadrequesting (through HTTP GET requests)CollaborationHandlerservlet to provide the latest application informations	Mainthread, a Thread polling for globalUpdates from the Application which callsGlobalUpdateHandlerwhich in turn queues the message received from the application inCollaborationHandler	At each iteration send out global updates to the middleware
ii - Issue requests	A request (HTTP POST) initiated from the Clientobject list to send a message to theCommandHandlerservlet, the command is added to the commandTable inApplication if not already there	CommandHandlerbuilds a request that is passed on to the back-end application as a String throughCommandConnection	Once the application processes the request it is sent back to the gateway and then picked by theResponseThreadwhich builds aResponse Object and queues it until theclientresponsethreadissues a GET request to theCollaborationHandler
iii - Enable/disable collaboration	Uesrs can control whether they want to receive other users views by checking or unchecking a checkbox inClientResponseThreadthat triggers a GET request toCollaborationHandler	The ClientID is obtained from the TableEntriesand the collaboration mode is switched, a string is sent back to the client to confirm the change of status	Noop
iv - Chat/Whiteboard	The Client class allows starting and closing a Chat and Whiteboard frame; once a user acts on those components, a POST request is passed on to the CollaborationHandler	An InfoPacket object is then created from the parameters posted describing the event and this object is queued. It is retrieved from the threaded GET requests done byClientResponseThreadtoCollaborationHandlerwhich deQueues Objects that are ready	Noop

INFORMATION SHARING IN A COLLABORATIVE ENVIRONMENT

The VizWiz [1] is one of the pioneer works, which provides remote visualization using data repository. It is implemented as a Java applet, which can be loaded and run automatically by a Web browser.

Another approach to Web collaboratory is the NCSA Habenero [2] written in Java. The system is based on a centralized server and right now only provides support for Java applications which causes high interconnect delays.

Collaborative Visualization [3] architecture develops the collaborative visualization system as an extension of IRIS explorer. The starting point is the architecture of a modular visualization environment, each module of the pipeline can be either local or from a session participant. A tradeoff has been made in terms of speed by using TCPIP to achieve the reliability of a dedicated connection for each participant.

The Tango system [4] was designed to have a centralized server and is web browser enabled using the LiveConnect interface (written in C) by Netscape. Separation of the messaging layer into control and application messaging provides a clean API, enabling existing applications to be integrated to this system by adhering to the application specific communication protocol. The system also provides dynamic entry/exit and rollback feature by using the central data repository to store the entire session.

The CCASEE [5] aims at providing a distributed workspace using Java RMI. The new object is detected by the participants by polling operation, and an update is made to the workspace and the JDBC based data repository. Each communication between the application and the client needs to be expressed in an intermediate format by building on existing libraries provided by ORBs, and thus is not feasible in a real time environment. Java RMI introduces inherent delays of its own due to the additional compilation required using rmic and lookup time.

The Java Environment for Distributed Invocation (JEDI) [6] provides an alternative to using RMI for dynamic invocation. Instead of involving a stub compiler, JEDI provides a software library for making remote method calls. The system does not support asynchronous method calls (with an option of receiving the return value later), but reduces the inherent latency involved with RMI.

The CEV system [7] is an approach at collaborative visualization using the central server to perform the computations necessary to generate new views. The server decides whether to send back the image using RMI or send it as a module used to perform the necessary image generation. The browser environment provides for easy interaction and reduces the resource requirement at the local machine. Control mechanism currently implemented uses the round robin algorithm with each participant allowed steering the application for a fixed time slice.

As for the Infospheres [8], the focus of this project is in investigating compositional systems supporting peer-to-peer communications among persistent, multithreaded, distributed objects. This framework is composed of two units, dapplets (which are multithreaded, communicating objects) and sessions (which groups of composed dapplets). The initial implementation of the system was using UDP socket, with an intermediate layer to ensure reassembly in the correct order at the receiver. Though this framework is highly generic, it is difficult to allow collaboration of legacy systems using this framework. Thus its application to the scientific computing scenario could be limited to new versions of the systems.

 

The Collaborative Virtual Workspace [9] utilizes a client/server architecture to implement the shared virtual workspace. It provides for text based interaction using TCPIP socket communication and audio/videoconferencing using the IP multicast.

 

References:

1. Cherilyn Michaels. "VizWiz: A Java applet for interactive 3D scientific visualization on the web" Technical report, university of California, San Diego.
2. Brent Driggers, Jay Alameda, Ken Bishop, "Distributed Collaboration for Engineering and Scientific Applications Implemented in Habanero, a Java-Based Environment"
3. Jason Wood, Helen Wright, Ken Brodie, " Collaborative Visualization", University of Leeds, UK.
4. Lukasz Beca, Gang Cheng, Geoffrey C. Fox, Tomasz Jurga, Konrad Olszewski, Marek Podgorny, Piotr Sokolowski, and Krzysztof Walczak, "Java enabling Collaborative Education, Healthcare and Computing", Syracuse university, New York.
5. Rajeev R. Raje, Antony Teal, Joeseph Coulson ,Sihai Yao and William Winn, " CCASEE- A Collaborative Computer Assisted Software Engineering Environment", Purdue University, Indianapolis.
6. Jonathan Aldrich, James Dooley, Scott Mandelsohn and Adam Rifkin, " Providing Easier Access to Remote Objects in Client- Server Systems", California Institute of Technology.
7. Michael Boyles, Rajeev Raje and Shiaofen Fang, "CEV: Collaboration Environment for Visualization Using Java RMI", Purdue University, Indianapolis.
8. K. Mani Chandy, Joseph Kiniry, Adam Rifkin and Daniel Zimmerman, "A Framework for Structured Distributed Object Computing", California Institute of Technology.
9. "Collaborative Virtual Workspace", The MITRE Corporation.

On Interactive Computational Steering

Introduction

Computational steering is the online management of the execution of an application and of the resources it uses. Such a monitoring tool is necessary for controlling the execution of long-running, resource intensive applications and for purposes of experimenting with application parameters or improving application performance.

In an interactive computational process, a sequence of specification, computation and analysis is performed. Interactive simulations is a way to increase the utility of high performance simulations whereby scientists can drive the scientific discovery process and interact with the data being computed. They can interpret what is happening to data during simulations and steer calculations at real-time. Also, computational steering enhances productivity by reducing the time between changes to parameters and the viewing of results. It enables the user to explore a what-if analysis, and as changes in parameters become more instantaneous, the cause-effect relationships become more evident.

The main areas where computational steering can be used can be distinguished as:

• model exploration,
• algorithm experimentation and
• performance optimization.

In model exploration, computational steering is used to explore simulation behavior to gain additional insight into the simulation. In algorithm experimentation, the users are allowed to adapt the program algorithms at run time. For example, a user can experiment with different methods to solve a partial differential equation. In performance optimization, steering is used to manually improve an application's performance, for example, to perform load balancing interactively in parallel applications.

Several computational steering environments have been built over the years. The results can be classified into application specific computational steering systems, domain specific computational steering systems and generic computational steering systems.

Characteristics of a Computational Steering System

The steering environment comprises of three major components: the user interface, the application, and the communication between the user interface and the application. These components may be handled by separate processes running in a distributed environment, or they can be executing in the same address space as the application. Also, the steering process does not have to be limited to single applications or single users. Several different users might perform collaborative steering of a single or multiple applications simultaneously.

The system must set up an interactive session between the application(s) and the users. Once this is accomplished, at every step of the application’s computation, the system should wait to see if any user requests are pending. If so, the desired information must be extracted to be presented to the user. The type of information can vary depending on the kind of the steering application. For example, in parallel or distributed applications, the information about the distribution of the data over different processors or platforms has to be known, along with processor and network loads. To actuate steering actions, the environment must have access to the application's data and the modules to modify these data. The access to the monitoring information and the steering items could be synchronous or asynchronous.

Another crucial aspect of the system is the user interface. The user interface has to present the information extracted from the application in an appropriate manner. This task could range from simple textual displays to complicated 3-D plots to visualize the data sets churned out by the application. Another important feature is to allow users to manipulate the steerable items of the application. This varies from application parameter manipulation for model exploration, to code generation for algorithm experimentation, and reconfiguration for performance optimization. Further, support must be available at the user end to allow the user to participate in an interactive session in collaboration with other users.

In order to compare existing computational steering systems, we need to group the aspects of the steering system into categories so that they can be compared based on a particular characteristic of a typical computational steering system.

This survey attempts to compare steering systems based on the following features / categories:

1) System scope

2) System Architecture

3) User interface

System scope

This category broadly comprises of the following features that are to be considered for evaluating a computational steering system.

a) Type of steering - Is the system designed for model exploration, algorithm experimentation, performance optimization, or a combination of these?

b) Ease of incorporating an existing application - Can an existing application be incorporated in the interactive framework or is the system designed for building entirely new applications?

c) Steering distributed applications - Can the system be distributed with the user interface and the scientific application running on different hosts?

d) Support for multiple applications - Can the framework be used for interacting with multiple applications simultaneously?

e) Multiple clients - Can multiple clients participate in a single interactive session with an application?

f) Portability - Can the system or components of the system be ported onto a different host?

g) Synchronous / Asynchronous access of data - Does the application permit the user to access data elements synchronously or asynchronously?

h) Fault - tolerant - Is the system fault tolerant? That is, if any one of the components fail, can the interaction resume/restore to original state?

System Architecture

This category attempts to categorize the diverse steering systems based on their architecture - this broadly takes into account the type of data/control flow between the various components of the system, data acquisition methods incorporated, etc.

a) Application annotation and Data Abstraction - To create a steering application or to enable steering in an application, it has to be indicated which information is to be provided to the user and which steerable items the user can manipulate.

b) Data Extraction - How does the system extract the monitoring system from the application?

c) Steering access - How does the system access the steerable items of an application?

d) Synchronization - What method of synchronization are available for the interaction process?

User Interface

This category identifies the ideal features of the interface provided to a user for steering the target application.

a) Data presentation - How is the monitoring information presented to the user?

b) Steering features - What kind of interface concepts are available to the user for manipulating the steerable items/application?

c) Collaboration - Can multiple clients collaborate in an interactive session with the application?

d) Customization of UI - can the user customize the UI according to his/her preferences?

 

Computational Steering Environment (CSE)

The CSE has been developed at the Center for mathematics and Computer Science in Amsterdam, The Netherlands.

System Scope

The steering capability of the CSE is focused at model exploration. Multiple existing applications can be integrated into the framework by annotation of the application source code. The applications and multiple user interfaces processes can be distributed over different platforms. The CSE is built on a blackboard architecture that is described in detail.

Blackboard models

In this model of problem solving, multiple independent agents are allowed to share information in a central shared space. The model serves as a schema for organizing reasoning steps and domain knowledge to construct a solution to a particular problem. Knowledge is segmented into modules and a separate inference engine is provided for each module. Communication between modules is realized by reading and writing in the blackboard. The blackboard model does not explicitly specify a control component. The actual control can be in the information modules, in the blackboard itself, in a separate module, or in a combination of these.

Architecture

The architecture of the CSE is built around a data manager that acts as a blackboard for communicating data values. Separate processes, called satellites, can connect to the data manager and exchange data with it. An application is packaged as a satellite. The application code is annotated to make a connection to the data manager, to publish its output variables for visualization and its input parameters for steering, an to indicate when these are to be read and / or written.

These satellites implement the simulation, analysis programs, and other visualization and rendering algorithms. The blackboard has two main functions.

• Manage a database of variables
• It acts as an event notification manager. Satellites can subscribe to events that represent state changes in the blackboard. Whenever such an event occurs, it will update the event to all subscribing satellites.

Variable – the basic blackboard object

Variables are defined as a tuple consisting of four components:

• Name
• Type Descriptor
• Raw data
• List of attributes

The name of an object is unique across all application objects. The type descriptor determines the size, type and layout of the data. The data component is the storage container for the raw data. Attributes are name value pairs that may be used to describe the meta-data of the variable. We presume this to mean attributes like access modifiers (READ, WRITE, etc.)

Architectural components

• Global Name Manager (GNM)

The GNM maintains a list of all variables in the system and in which blackboard these variables exist. Only variable names, descriptors and attributes are stored – not the raw data.

• Local Blackboard

A local blackboard resides on each host in the distributed environment. Local blackboards accept connections from satellites executing on the same host and other LBBs. Variable data is stored in the LBB, and is shared by all connecting satellites.

• Satellites

A satellite is a process that communicates with its corresponding LBB. Satellites may create, open, close, read/write, and subscribe to variable events.

Programming Abstractions

CSE provides a three-layered API to access the LBB, each layer providing a higher level of abstraction than the other.

The local blackboard API is a low level library interface that provides functionality for LBB communication, variable management and event management. This layer provides all details of the underlying LBB protocols. It is difficult to use due to the inherent parallelism in the system.

The higher layer provides a better interface to the LBB. This library hides the notion of events and has built in support for structuring variables into sets, and support for handling efficient set I/O.

User Interface

The user interface is by means of a satellite called the PGO Editor. It is an interactive graphic tool that allows users to create custom 2D, 3D user interfaces to visualize and manipulate data in the data manager. The user draws an initial layout of the interface with graphic objects. He then specifies how the geometry and attributes of these objects should be coupled to variables present in the data manager. The coupling of these objects can be bi-directional. As a result, the user can manipulate variables by direct manipulation on the object.

Summary

The architecture has been implemented on a number of heterogeneous UNIX and NT machines. TCP/IP is used for across host communication and, when possible, shared memory for satellite to LBB communication. Viewers are available for many displays, ranging from low-end laptop displays to sophisticated VR displays like CAVE.

Important features

• The variable naming mechanism is used to bind data structures in the blackboard to data structures in the satellite. The event mechanism is used to maintain the consistency of these data structures. As a result of this, a viewing satellite can directly manipulate data structures of other satellites.
• Efficiency of data transport: Distributed blackboards maintain local copies of a data structure. An event is broadcast when a satellite mutates the data structure. The data structure is transported to other LBBs only when it is needed. This saves bandwidth.
• Easy-to-use programming abstractions.

 

Visualization and Application Steering Environment (VASE)

VASE was one of the first computational steering systems built, and was developed from the University of Illinois.

Features

• Designed for algorithm experimentation and model exploration.
• Steering capabilities include altering of key parameters and addition of new code at key places.
• Existing applications can be integrated by manual annotation of application source code. An abstract program structure is created that describes the essential algorithms and data on a high level.
• Multiple applications can be steered simultaneously.
• No collaboration support among multiple clients.

Architecture

• System is a collection of programming tools and system software.
• Monitoring and steering are performed by breakpoint scripts. Scripts can read and write variables in the application, control the flow of data between processes and call subroutines defined in the application.
• Scripts are executed by the VASE interpreter that is linked directly to the application and therefore has access to the application’s address space.

User Interface

• Configuration includes adding or deleting applications, specifying application hosts, altering breakpoint scripts, etc.
• Actual steering is textual using breakpoint scripts.
• No tools for visualization – have to use existing packages.

Progress and Magellan

Features

• Designed for model exploration and performance optimization.
• Existing applications can be integrated by manual annotation of application source code. Provides end user with an abstraction of the application that only reveals the important steering parameters and output data.
• Progress does not allow steering multiple applications simultaneously. Magellan has this capability.
• No collaboration support among multiple clients.

Architecture

• The application’s code is annotated with Progress’s library calls to create a steering object model.
• The operations probe(an asynchronous read or write), sense (a synchronous modification of an object’s state), and actuate ( a synchronous modification of a steering object) can be performed on all objects.
• Synch points, functions and scripts are operations that can be performed on specialized steering objects. Synch points are used to halt an application. Functions are for modifying application specific data structures. Scripts provide users with the functionality of combining other steering operations in a language form.

User Interface

 

 

• No tools for visualization – have to use existing packages.

 

Comparison of computational steering environments

 

Feature\ System	Remarks	CSE	Progress & Magellan	VASE	SCIRun	CUMULVS	VIPER
Overall Scope
Model Exploration		C	C	C	C	C	C
Algorithm Experimentation				C	C
Performance Optimization			C			C
Steering capabilities			Language (ACSL) - directed steering. Focuses on steering latency and application perturbation	Altering of key parameters Addition of new code at key places
Support for existing applications	Can existing applications be incorporated into steering framework?	Annotation of application source code needed	Manual annotation of application code needed	Manual annotation of application source code needed
Remote Steering capability	Can application and steering system execute on different hosts?		Steering clients can run on a different host
Support for steering multiple applications	Can the system be used for steering multiple applications simultaneously?		Progress : NO Magellan : YES	Multiple applications can run & steered.
Support for multiple clients	Can multiple clients collaborate in an interactive session?		No support for collaboration	No support for collaborative steering by multiple users
Support for Fault-tolerance			Yes, in the sense that there are multiple servers running in the same address space as the application	No fault – tolerance
Architectural Issues
Application Integration			A steering object model is created using the system’s library calls.	Added annotations creates abstract steerable program structure
Data and Parameter Access Model			Servers can access data asynchronously.	Hybrid Control and Data Flow
Portability	Is the system or any of its’ components portable?		UNKNOWN Client is a Motif application – can run on remote machine	UNKNOWN
Synchronization Support			Synchronous sense and actuate operations	Synchronous breakpoints in source code
Data Abstraction & Communication	Ease of programming		Communication between steering clients and servers is based on ACSL.	Communication over data channels between VASE data ports.
User Interface
Data Presentation			Visualization through existing packages.	Visualization through existing packages. No in-built tools.
Steering Support			Steering through Tcl/Tk interfaces and widgets.	Steering in textual manner using breakpoint scripts – written in extended C, interpreted at run time
Support for UI customization			Command line and GUI for end–user. Limited graphical display.	None