Warning

This version of easy.gems was built within branch remove-memusage-tip for revision b94a8064. You can find the most recent release version of easy.gems at https://easy.gems.dkrz.de/.

Working with Intake and Intake-ESM

Intake is a library that allows you to easily access data from a variety of sources in a tree-like catalog structure.

Its sub-variant ,`Intake-ESM <https://intake-esm.readthedocs.io/en/latest/>`__ assumes a table/database style structure with grouping by pre-specified columns. It is mainly used in the context of CMIP, where data is strongly standardized, and thus many models can be compared easily, and intake scripts can iterate across models and ensemble members.

Tutorials for Intake

• Listing the entries in an intake catalog
• Finding files on the command line with query_yaml
• Searching catalog metadata with search_yaml

Tutorials for Intake-ESM

• Loading data from the catalog
• Working with Intake-ESM and the Tape archive
• Map Plotting with Intake and datashader
• Caching results of computations for later use
• Accessing the metadata of variables
• Calculating zonal means
• Using intake together with CDO

Utilities

• Getting file names from intake with a command-line python script
  • Getting the full details of a dataset
  • Controlling the output format
  • Using time ranges
  • Using find_files to download data from the tape archive
  • Using find_files with cdo
  • The code
• Some helper functions

Special cases:

• Robust Script for getting variable metadata
• Plotting 2m air temperature for all DYAMOND WINTER simulations

Further reading

• The official Intake-ESM documentation

• The Intake-ESM tutorials of DKRZ’s Data Management team

• The Intake-ESM Tech Talk by Aaron Spring