AtmoRep: Foundation Model for Earth System Science

AtmoRep is a novel, task-independent stochastic machine learning model of atmospheric dynamics that uses large-scale representation learning to provide skillful results for a wide range of Earth system science applications. Trained on 30 years of ERA5 data, AtmoRep employs a self-supervised approach with large-scale transformer networks to learn a general description of the complex, stochastic dynamics of the atmosphere. For further details, refer to AtmoRep.

This hands-on hackathon aims to introduce researchers in the Earth system science domain to get familiar with the AtmoRep codebase and facilitate them to apply it to their applications. The program consists of a series of small introductory lectures and hands-on exercises covering the following topics:

1. Getting started with the JUWELS booster supercomputer
2. Understanding attention mechanisms, the core component of transformers
3. Working with Zarr data formats
4. Introduction to AtmoRep
5. Understanding the data pipeline for AtmoRep
6. Exploring the evaluation script
7. Running  the training script
8. Applying AtmoRep to a downstream task
9. Experimenting with alternative masking strategies

By the end of this hackathon, participants will have a strong understanding of AtmoRep’s capabilities and will be able to utilize this powerful foundation model for their own Earth system science research and applications.