In Situ Analysis and Visualization with ParaView Catalyst and Ascent

Course description

Scientific visualization and analysis are key ingredients in HPC simulation workflows. For decades, the dominant paradigm has been post-hoc visualization; simulation codes iterate and save files to disk, giving the domain scientists the opportunity to read the data back at a later time for analysis. In recent years though, this paradigm has been stressed by an ever-diverging rate of growth between I/O and compute speeds. In-situ processing helps mitigate these I/O bottlenecks, enabling simulation and visualization calculations to run in-memory, at higher spatial and temporal resolution, avoiding the transfer of raw data to disks.

This half-day tutorial introduces ParaView Catalyst and Ascent, two open-source implementations enabling in-situ processing. Both packages share a common project, called Conduit, which provides an intuitive model for describing hierarchical scientific data in C++, C, Fortran, and Python. We will cover how to describe simulation data with Conduit and how Ascent or Catalyst can transform data, render images, and export results, discussing the pros and cons of both implementations.

Targeted audience

Domain scientists and code developers and visualization practitioners in HPC will benefit from this tutorial by understanding the needs and advantages of integrating an in-situ visualization support into their code. They will have a brief overview of the current landscape of in-situ processing solutions, before focusing on two specific implementations. No particular knowledge of the implementation details of the data model used by the two libraries are required to follow the tutorial and to adopt the concepts learned. The two environments described are part of very active project developments by large teams with committed resources for future developments. Attendees can follow along, execute some tutorial examples using Jupyter Notebooks, and deploy two example mini-apps (mesh- and particle-based) to illustrate the ideas presented. Examples will require elementary knowledge of Python and C++.

Agenda

13:00 Welcome, Overview and Motivation

Agenda and technical details for demonstrations

13:15 Introduction to in-situ visualization, workflows and terminology

13:30 Conduit, an API to describe hierarchical scientific data

The Mesh Blueprint, usage conventions, examples

14:00 Ascent, an in-situ visualization and analysis library using Conduit

Making images
Transforming data, extracting data
Queries and Conditional triggers

14:30 Instrumentation of two proxy Apps with Ascent

SPH-EXA: an open-source parallel particle simulation code
LULESH: an open-source parallel unstructured flow solver
Compilation and runtime details

15:00 Coffee break

15:30 ParaView Catalyst

The ParaView interactive application, introduction
The Paraview parallel server architecture
The Catalyst API
The ParaView Catalyst Blueprint
In-situ Python API, Data Extractors

16:00 Instrumentation of two proxy Apps with Catalyst

SPH-EXA: an open-source parallel particle simulation code
LULESH: an open-source parallel unstructured flow solver
Compilation and runtime details
Catalyst, connecting to a live simulation, steering
ParaView Cinema: An Image-Based Approach to Extreme-Scale Data Analysis

16:25 Ascent executing ParaView Python code

16:35 Future developments, alternative workflows (in-transit visualization)

16:45 Wrap-up, Q&A

17:00 End

Instructor

Dr. Jean Favre (Senior Visualisation Software Engineer, CSCS)

Registration

All participants must register for the course. The registration fee includes a coffee break.

Course Fee: 30 CHF

Deadline for registration: Sunday, May 1, 2022

Kindly note that the course can take place only if there are sufficient confirmed registrations received by the deadline. The minimum number of participants is 8.

Participant's experience

The ventilation of the meeting room is in line with the Swiss standards (5 full renewals of fresh air in the space per hour).

There will be no physical distancing between seats in the meeting room, nor obligation to wear a mask. You are invited to wear a mask if that makes you feel more comfortable.

On April 1, 2022, the Federal Council has decided to revoke all COVID-19 measures against the pandemic. Now that the measures and restrictions related to events have been lifted, individual responsibility in protecting everyone's health has become of primary importance.

We ask all participants to follow the federal recommendations regarding hygiene and behaviour in a pandemic situation:

Cough and sneeze into a tissue or the crook of your arm
Wash or sanitise your hands thoroughly (At the entrance of the building, you will find hand sanitizer for your use. We warmly recommend regular use of the hand sanitizer or regularly cleaning hands in the washrooms.)

If you are feeling unwell, we ask you to please write an email to reception@cscs.ch and explain that you are unable to attend due to a suspect case of coronavirus infection. Your registration fee will then be reimbursed. Most common coronavirus symptoms are:

Symptoms of a respiratory illness (sore throat, cough (usually dry), shortness of breath, chest pain)
High temperature
Sudden loss of sense of smell and/or taste
Headache
General weakness, feeling unwell
Aching muscles
Gastrointestinal symptoms (nausea, vomiting, diarrhoea, stomach ache)
Head cold
Skin rash

Should the epidemiologic situation change and require the introduction of new regulations, we will promptly inform registered participants.

Contacts

Please contact Jean Favre (jean.favre@cscs.ch) for questions related to the course content and reception@cscs.ch for questions related to the event logistics.

Directions

Kindly note that no parking space is available at the Swiss National Supercomputing Centre. There is a Park & Ride Resega parking lot, within five minutes walk from CSCS.

You are encouraged to travel by public transportation. The closest bus stop to CSCS is "Lugano, Stadio." From Lugano railway station, you should take bus number 4 or 6.

Suggestions regarding travels and accommodation are available here.