{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "user_expressions": []
   },
   "source": [
    "# ICESat-2's Nested Variables\n",
    "\n",
    "This notebook ({nb-download}`download <IS2_data_variables.ipynb>`) illustrates the use of icepyx for managing lists of available and wanted ICESat-2 data variables.\n",
    "The two use cases for variable management within your workflow are:\n",
    "1. During the data access process via remote cloud access.\n",
    "2. When reading in data to a Python object (whether from local files or the cloud).\n",
    "\n",
    "A given ICESat-2 product may have over 200 variable + path combinations.\n",
    "icepyx includes a custom `Variables` module that is \"aware\" of the ATLAS sensor and how the ICESat-2 data products are stored.\n",
    "The module can be accessed independently and can also be accessed as a component of a `Query` object or `Read` object.\n",
    "\n",
    "This notebook illustrates in detail how the `Variables` module behaves. We use the module independently and also show how powerful it is directly in the icepyx workflow using a `Query` data access example.\n",
    "Module usage using `Query` is analogous through an icepyx ICESat-2 `Read` object.\n",
    "More detailed example workflows specifically for the [query](https://icepyx.readthedocs.io/en/latest/example_notebooks/IS2_data_access.html) and [read](https://icepyx.readthedocs.io/en/latest/example_notebooks/IS2_data_read-in.html) tools within icepyx are available as separate Jupyter Notebooks.\n",
    "\n",
    "Questions? Be sure to check out the FAQs throughout this notebook, indicated as italic headings."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "user_expressions": []
   },
   "source": [
    "### _Why do ICESat-2 products need a custom variable manager?_\n",
    "\n",
    "_It can be confusing and cumbersome to comb through the 200+ variable and path combinations contained in ICESat-2 data products._\n",
    "_An hdf5 file is built like a folder with files in it. Opening an ICESat-2 file can be like opening a new folder with over 200 files in it and manually searching for only ones you want!_\n",
    "\n",
    "_The icepyx `Variables` module makes it easier for users to quickly find and extract the specific variables they would like to work with across multiple beams, keywords, and variables and provides reader-friendly formatting to browse variables._\n",
    "_A future development goal for `icepyx` includes developing an interactive widget to further improve the user experience._\n",
    "_For data read-in, additional tools are available to target specific beam characteristics (e.g. strong versus weak beams)._"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import packages, including icepyx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import icepyx as ipx\n",
    "from pprint import pprint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "user_expressions": []
   },
   "source": [
    "## Creating or Accessing ICESat-2 Variables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "user_expressions": []
   },
   "source": [
    "There are three ways to create or access an ICESat-2 Variables object in icepyx:\n",
    "1. Access via the `.variables` property of a Query object\n",
    "2. Access via the `.variables` property of a Read object\n",
    "3. Create a stand-alone ICESat-2 Variables object using a local file, cloud file, or a product name\n",
    "\n",
    "An example of each of these is shown below."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "user_expressions": []
   },
   "source": [
    "### 1. Access `Variables` via the `.variables` property of a Query object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "region_a = ipx.Query('ATL06',[-55, 68, -48, 71],['2019-02-22','2019-02-28'], \\\n",
    "                           start_time='00:00:00', end_time='23:59:59')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Showing the variable paths, prior icepyx versions utilized order_vars()\n",
    "region_a.variables.avail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": [],
    "user_expressions": []
   },
   "source": [
    "### 2. Access via the `.variables` property of a Read object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "path_root = '/full/path/to/your/data/'\n",
    "reader = ipx.Read(path_root)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Accessing Variables\n",
    "reader.variables.parse_var_list(region_a.variables.avail())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "user_expressions": []
   },
   "source": [
    "### 3. Create a stand-alone Variables object\n",
    "\n",
    "You can also generate an independent Variables object. This can be done using either:\n",
    "1. The filepath to a local or cloud file you'd like a variables list for\n",
    "2. The product name (and optionally version) of a an ICESat-2 product\n",
    "\n",
    "*Note: Cloud data access requires a valid Earthdata login; \n",
    "you will be prompted to log in if you are not already authenticated.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "user_expressions": []
   },
   "source": [
    "Create a variables object from a filepath:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "filepath = '/full/path/to/your/data.h5'\n",
    "v = ipx.Variables(path=filepath)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# v.avail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "user_expressions": []
   },
   "source": [
    "Create a variables object from a product. The version argument is optional."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "v = ipx.Variables(product='ATL03')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "v.avail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "v = ipx.Variables(product='ATL03', version='006')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "v.avail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "user_expressions": []
   },
   "source": [
    "Now that you know how to create or access Variables, the remainder of this notebook showcases the functions available for building and modifying variables lists. Remember, the example shown below uses a Query object, but the same methods are available if you are using a Read object or a Variables object."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "user_expressions": []
   },
   "source": [
    "## Interacting with ICESat-2 Data Variables\n",
    "\n",
    "Each variables instance (which is actually an associated Variables class object) contains two variable list attributes.\n",
    "One is the list of possible or available variables (`avail` attribute) and is unmutable, or unchangeable, as it is based on the input product specifications or files.\n",
    "The other is the list of variables you'd like to actually have (in your downloaded file or data object) from all the potential options (`wanted` attribute) and is updateable.\n",
    "\n",
    "Thus, your `avail` list depends on your data source and whether you are accessing or reading data, while your `wanted` list may change for each analysis you are working on or depending on what variables you want to see.\n",
    "\n",
    "The variables parameter has methods to:\n",
    "* get a list of all available variables from the file (`avail()` method).\n",
    "* append new variables to the wanted list (`append()` method).\n",
    "* remove variables from the wanted list (`remove()` method).\n",
    "\n",
    "We'll showcase the use of all of these methods and attributes below using an `icepyx.Query` object.\n",
    "Usage is identical in the case of an `icepyx.Read` object.\n",
    "More detailed example workflows specifically for the [query](https://icepyx.readthedocs.io/en/latest/example_notebooks/IS2_data_access.html) and [read](https://icepyx.readthedocs.io/en/latest/example_notebooks/IS2_data_read-in.html) tools within icepyx are available as separate Jupyter Notebooks.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "user_expressions": []
   },
   "source": [
    "Create a query object and log in to Earthdata\n",
    "\n",
    "For this example, we'll be working with a land ice product (ATL06) for an area along West Greenland (Disko Bay).\n",
    "A second option for an atmospheric product (ATL09) that uses profiles instead of the ground track (gt) categorization is also provided."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a = ipx.Query('ATL06',[-55, 68, -48, 71],['2019-02-22','2019-02-28'], \\\n",
    "                           start_time='00:00:00', end_time='23:59:59')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # Uncomment and run the code in this cell to use the second variable subsetting suite of examples,\n",
    "# # with the beam specifier containing \"profile\" instead of \"gt#l\"\n",
    "\n",
    "# region_a = ipx.Query('ATL09',[-55, 68, -48, 71],['2019-02-22','2019-02-28'], \\\n",
    "#              start_time='00:00:00', end_time='23:59:59')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "user_expressions": []
   },
   "source": [
    "### ICESat-2 data variables\n",
    "\n",
    "ICESat-2 data is natively stored in a nested file format called hdf5.\n",
    "Much like a directory-file system on a computer, each variable (file) has a unique path through the hierarchy (directories) within the file.\n",
    "Thus, some variables (e.g. `'latitude'`, `'longitude'`) have multiple paths (one for each of the six beams in most products).\n",
    "\n",
    "#### Determine what variables are available\n",
    "`region_a.variables.avail` will return a list of all valid path+variable strings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "region_a.variables.avail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "user_expressions": []
   },
   "source": [
    "To increase readability, you can use built in functions to show the 200+ variable + path combinations as a dictionary where the keys are variable names and the values are the paths to that variable.\n",
    "`region_a.variables.parse_var_list(region_a.variables.avail())` will return a dictionary of variable:paths key:value pairs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "region_a.variables.parse_var_list(region_a.variables.avail())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "user_expressions": []
   },
   "source": [
    "By passing the boolean `options=True` to the `avail` method, you can obtain lists of unique possible variable inputs (var_list inputs) and path subdirectory inputs (keyword_list and beam_list inputs) for your data product. These can be helpful for building your wanted variable list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "region_a.variables.avail(options=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "user_expressions": []
   },
   "source": [
    "```{admonition} Remember\n",
    "You can run these same methods no matter how you created or accessed your ICESat-2 Variables. So the methods in this section could be equivalently be accessed using a Read object, or by directly accessing a file on your computer:\n",
    "\n",
    "```\n",
    "```python\n",
    "# Using a Read object\n",
    "reader.variables.avail()\n",
    "reader.variables.parse_var_list(reader.variables.avail())\n",
    "reader.variables.avail(options=True)\n",
    "\n",
    "# Using a file on your computer\n",
    "v = Variables(path='/my/file.h5')\n",
    "v.avail()\n",
    "v.parse_var_list(v.avail())\n",
    "v.avail(options=True)\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Building your wanted variable list\n",
    "\n",
    "Now that you know which variables and path components are available, you need to build a list of the ones you'd like included.\n",
    "There are several options for generating your initial list as well as modifying it, giving the user complete control.\n",
    "\n",
    "The options for building your initial list are:\n",
    "1. Use a default list for the product (not yet fully implemented across all products. Have a default variable list for your field/product? Submit a pull request or post it as an issue on [GitHub](https://github.com/icesat2py/icepyx)!)\n",
    "2. Provide a list of variable names\n",
    "3. Provide a list of profiles/beams or other path keywords, where \"keywords\" are simply the unique subdirectory names contained in the full variable paths of the product. A full list of available keywords for the product is displayed in the error message upon entering `keyword_list=['']` into the `append` function (see below for an example) or by running `region_a.variables.avail(options=True)`, as above.\n",
    "\n",
    "**Note: all products have a short list of \"mandatory\" variables/paths (containing spacecraft orientation and time information needed to convert the data's `delta_time` to a readable datetime) that are automatically added to any built list. If you have any recommendations for other variables that should always be included (e.g. uncertainty information), please let us know!**\n",
    "\n",
    "Examples of using each method to build and modify your wanted variable list are below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.wanted"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.append(defaults=True)\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "user_expressions": []
   },
   "source": [
    "The keywords available for this product are shown in the error message upon entering a blank keyword_list, as seen in the next cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.append(keyword_list=[''])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Modifying your wanted variable list\n",
    "\n",
    "Generating and modifying your variable request list, which is stored in `region_a.variables.wanted`, is controlled by the `append` and `remove` functions that operate on `region_a.variables.wanted`. The input options to `append` are as follows (the full documentation for this function can be found by executing `help(region_a.variables.append)`).\n",
    "* `defaults` (default False) - include the default variable list for your product (not yet fully implemented for all products; please submit your default variable list for inclusion!)\n",
    "* `var_list` (default None) - list of variables (entered as strings)\n",
    "* `beam_list` (default None) - list of beams/profiles (entered as strings)\n",
    "* `keyword_list` (default None) - list of keywords (entered as strings); use `keyword_list=['']` to obtain a list of available keywords\n",
    "\n",
    "Similarly, the options for `remove` are:\n",
    "* `all` (default False) - reset `region_a.variables.wanted` to None\n",
    "* `var_list` (as above)\n",
    "* `beam_list` (as above)\n",
    "* `keyword_list` (as above)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.remove(all=True)\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Examples (Overview)\n",
    "Below are a series of examples to show how you can use `append` and `remove` to modify your wanted variable list.\n",
    "For clarity, `region_a.variables.wanted` is cleared at the start of many examples.\n",
    "However, multiple `append` and `remove` commands can be called in succession to build your wanted variable list (see Examples 3+).\n",
    "\n",
    "There are two example tracks.\n",
    "The first is for land ice (ATL06) data that is separated into beams.\n",
    "The second is for atmospheric data (ATL09) that is separated into profiles.\n",
    "Both example tracks showcase the same functionality and are provided for users of both data types."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------------------\n",
    "### Example Track 1 (Land Ice - run with ATL06 dataset)\n",
    "\n",
    "#### Example 1.1: choose variables\n",
    "Add all `latitude` and `longitude` variables across all six beam groups. Note that the additional required variables for time and spacecraft orientation are included by default."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.append(var_list=['latitude','longitude'])\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 1.2: specify beams and variable\n",
    "Add `latitude` for only `gt1l` and `gt2l`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.remove(all=True)\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "var_dict = region_a.variables.append(beam_list=['gt1l', 'gt2l'], var_list=['latitude'])\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 1.3: add/remove selected beams+variables\n",
    "Add `latitude` for `gt3l` and remove it for `gt2l`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.append(beam_list=['gt3l'],var_list=['latitude'])\n",
    "region_a.variables.remove(beam_list=['gt2l'], var_list=['latitude'])\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 1.4: `keyword_list`\n",
    "Add `latitude` and `longitude` for all beams and with keyword `land_ice_segments`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.append(var_list=['latitude', 'longitude'],keyword_list=['land_ice_segments'])\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 1.5: target a specific variable + path\n",
    "Remove `gt1r/land_ice_segments/longitude` (but keep `gt1r/land_ice_segments/latitude`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.remove(beam_list=['gt1r'], var_list=['longitude'], keyword_list=['land_ice_segments'])\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 1.6: add variables not specific to beams/profiles\n",
    "Add `rgt` under `orbit_info`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.append(keyword_list=['orbit_info'],var_list=['rgt'])\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 1.7: add all variables+paths of a group\n",
    "In addition to adding specific variables and paths, we can filter all variables with a specific keyword as well. Here, we add all variables under `orbit_info`. Note that paths already in `region_a.order_vars.wanted`, such as `'orbit_info/rgt'`, are not duplicated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.append(keyword_list=['orbit_info'])\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 1.8: add all possible values for variables+paths\n",
    "Append all `longitude` paths and all variables/paths with keyword `land_ice_segments`.\n",
    "\n",
    "Similarly to what is shown in Example 4, if you submit only one `append` call as `region_a.variables.append(var_list=['longitude'], keyword_list=['land_ice_segments'])` rather than the two `append` calls shown below, you will only add the variable `longitude` and only paths containing `land_ice_segments`, not ALL paths for `longitude` and ANY variables with `land_ice_segments` in their path."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "region_a.variables.append(var_list=['longitude'])\n",
    "region_a.variables.append(keyword_list=['land_ice_segments'])\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 1.9: remove all variables+paths associated with a beam\n",
    "Remove all paths for `gt1l` and `gt3r`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "region_a.variables.remove(beam_list=['gt1l','gt3r'])\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 1.10: generate a default list for the rest of the tutorial\n",
    "Generate a reasonable variable list prior to download"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "region_a.variables.remove(all=True)\n",
    "region_a.variables.append(defaults=True)\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "------------------\n",
    "### Example Track 2 (Atmosphere - run with ATL09 dataset commented out at the start of the notebook)\n",
    "\n",
    "#### Example 2.1: choose variables\n",
    "Add all `latitude` and `longitude` variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.append(var_list=['latitude','longitude'])\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 2.2: specify beams/profiles and variable\n",
    "Add `latitude` for only `profile_1` and `profile_2`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.remove(all=True)\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "var_dict = region_a.variables.append(beam_list=['profile_1','profile_2'], var_list=['latitude'])\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 2.3: add/remove selected beams+variables\n",
    "Add `latitude` for `profile_3` and remove it for `profile_2`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.append(beam_list=['profile_3'],var_list=['latitude'])\n",
    "region_a.variables.remove(beam_list=['profile_2'], var_list=['latitude'])\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 2.4: `keyword_list`\n",
    "Add `latitude` for all profiles and with keyword `low_rate`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.append(var_list=['latitude'],keyword_list=['low_rate'])\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 2.5: target a specific variable + path\n",
    "Remove `'profile_1/high_rate/latitude'` (but keep `'profile_3/high_rate/latitude'`)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.remove(beam_list=['profile_1'], var_list=['latitude'], keyword_list=['high_rate'])\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 2.6: add variables not specific to beams/profiles\n",
    "Add `rgt` under `orbit_info`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.append(keyword_list=['orbit_info'],var_list=['rgt'])\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 2.7: add all variables+paths of a group\n",
    "In addition to adding specific variables and paths, we can filter all variables with a specific keyword as well. Here, we add all variables under `orbit_info`. Note that paths already in `region_a.variables.wanted`, such as `'orbit_info/rgt'`, are not duplicated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.append(keyword_list=['orbit_info'])\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 2.8: add all possible values for variables+paths\n",
    "Append all `longitude` paths and all variables/paths with keyword `high_rate`.\n",
    "Similarly to what is shown in Example 4, if you submit only one `append` call as `region_a.order_vars.append(var_list=['longitude'], keyword_list=['high_rate'])` rather than the two `append` calls shown below, you will only add the variable `longitude` and only paths containing `high_rate`, not ALL paths for `longitude` and ANY variables with `high_rate` in their path."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.append(var_list=['longitude'])\n",
    "region_a.variables.append(keyword_list=['high_rate'])\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 2.9: remove all variables+paths associated with a profile\n",
    "Remove all paths for `profile_1` and `profile_3`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.remove(beam_list=['profile_1','profile_3'])\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 2.10: generate a default list for the rest of the tutorial\n",
    "Generate a reasonable variable list prior to download"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "region_a.variables.remove(all=True)\n",
    "region_a.variables.append(defaults=True)\n",
    "pprint(region_a.variables.wanted)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using your wanted variable list\n",
    "\n",
    "**NOTE**: Variable subsetting is not yet available in Harmony for ICESat-2 products."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "user_expressions": []
   },
   "source": [
    "#### With a `Read` object\n",
    "Calling the `load()` method on your `Read` object will automatically look for your wanted variable list and use it.\n",
    "Please see the [read-in example Jupyter Notebook](https://icepyx.readthedocs.io/en/latest/example_notebooks/IS2_data_read-in.html) for a complete example of this usage.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "user_expressions": []
   },
   "source": [
    "#### With a local filepath\n",
    "\n",
    "One of the benefits of using a local filepath in variables is that it allows you to easily inspect the variables that are available in your file. Once you have a variable of interest from the `avail` list, you could read that variable in with another library, such as xarray. The example below demonstrates this assuming an ATL06 ICESat-2 file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# filepath = '/full/path/to/my/ATL06_file.h5'\n",
    "v = ipx.Variables(path=filepath)\n",
    "v.avail()\n",
    "# Browse paths and decide you need `gt1l/land_ice_segments/`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import xarray as xr\n",
    "\n",
    "xr.open_dataset(filepath, group='gt1l/land_ice_segments/', engine='h5netcdf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "user_expressions": []
   },
   "source": [
    "You'll notice in this workflow you are limited to viewing data only within a particular group. Icepyx also provides functionality for merging variables within or even across files. See the [read-in example Jupyter Notebook](https://icepyx.readthedocs.io/en/latest/example_notebooks/IS2_data_read-in.html) for more details about these features of icepyx."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Credits\n",
    "* based on the subsetting notebook by: Jessica Scheick and Zheng Liu"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}