In Situ Analysis and Visualization with ParaView Catalyst and Ascent

Course description

Scientific visualization and analysis are key ingredients in HPC simulation workflows. For decades, the dominant paradigm has been post-hoc visualization; simulation codes iterate and save files to disk, giving the domain scientists the opportunity to read the data back at a later time for analysis. In recent years though, this paradigm has been stressed by an ever-diverging rate of growth between I/O and compute speeds. In-situ processing helps mitigate these I/O bottlenecks, enabling simulation and visualization calculations to run in-memory, at higher spatial and temporal resolution, avoiding the transfer of raw data to disks.

This full-day tutorial introduces ParaView Catalyst and Ascent, two open-source implementations enabling in-situ processing. Both packages share a common project, called Conduit, which provides an intuitive model for describing hierarchical scientific data in C++, C, Fortran, and Python. We will cover how to describe simulation data with Conduit and how Ascent or Catalyst can transform data, render images, and export results, discussing the pros and cons of both implementations.

Target audience

Domain scientists and code developers and visualization practitioners in HPC will benefit from this tutorial by understanding the needs and advantages of integrating an in-situ visualization support into their code. They will have a brief overview of the current landscape of in-situ processing solutions, before focusing on two specific implementations. No particular knowledge of the implementation details of the data model used by the two libraries are required to follow the tutorial and to adopt the concepts learned. The two environments described are part of very active project developments by large teams with committed resources for future developments. Attendees can follow along, execute some tutorial examples using Jupyter Notebooks, and deploy two example mini-apps (mesh- and particle-based) to illustrate the ideas presented. Examples will require elementary knowledge of Python and C++.

Trainer

Dr. Jean M. Favre, Senior Visualization Software Engineer

Agenda

09:30 Welcome, Overview and Motivation

Agenda and technical details for demonstrations

09:45 Introduction to in-situ visualization, workflows and terminology

10:15 Conduit, an API to describe hierarchical scientific data

The Mesh Blueprint, usage conventions, examples

10:45 Coffee break

11:00 Ascent, an in-situ visualization and analysis library using Conduit

Making images
Transforming data, extracting data
Queries and Conditional triggers

11:30 Instrumentation of two proxy Apps with Ascent

SPH-EXA: an open-source parallel particle simulation code
LULESH: an open-source parallel unstructured flow solver
Compilation and runtime details

12:00 Lunch break

13:00 ParaView Catalyst

The ParaView interactive application, introduction
The Paraview parallel server architecture
The Catalyst API
The ParaView Catalyst Blueprint
In-situ Python API, Data Extractors

14:00 Instrumentation of two proxy Apps with Catalyst

SPH-EXA: an open-source parallel particle simulation code
LULESH: an open-source parallel unstructured flow solver
Compilation and runtime details
Catalyst, connecting to a live simulation, steering
ParaView Cinema: An Image-Based Approach to Extreme-Scale Data Analysis

15:00 Coffee break

15:30 Ascent executing ParaView Python code

16:00 Special topics, future developments, in-transit visualization

16:45 Wrap-up, Q&A

17:00 End

Registration

All participants must register for the course. The registration fee includes lunch and coffee breaks.

Course Fee: 80 CHF

Deadline for registration: Monday, May 1, 2023

Kindly note that the course can take place only if there are sufficient confirmed registrations received by the deadline. The minimum number of participants is 8. The maximum number of participants is 23.

Contacts

Please contact Jean Favre (jean.favre@cscs.ch) for questions related to the course content and reception@cscs.ch for questions related to the event logistics.

Directions

Kindly note that no parking space is available at the Swiss National Supercomputing Centre. There is a Park & Ride Resega parking lot, within five minutes walk from CSCS.

You are encouraged to travel by public transportation. The closest bus stop to CSCS is "Lugano, Stadio." From Lugano railway station, you should take bus number 4 or 6.

Suggestions regarding travels and accommodation are available here.