{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Introduction to loading data \n", "\n", "* **[Sign up to the DEA Sandbox](https://app.sandbox.dea.ga.gov.au/)** to run this notebook interactively from a browser\n", "* **Compatibility:** Notebook currently compatible with both the `NCI` and `DEA Sandbox` environments\n", "* **Products used:** \n", "[ga_ls7e_nbart_gm_cyear_3](https://explorer.dea.ga.gov.au/ga_ls7e_nbart_gm_cyear_3),\n", "[ga_ls8c_nbart_gm_cyear_3](https://explorer.dea.ga.gov.au/ga_ls8c_nbart_gm_cyear_3)\n", "* **Prerequisites:** Users of this notebook should have a basic understanding of:\n", " * How to run a [Jupyter notebook](01_Jupyter_notebooks.ipynb)\n", " * The basic structure of the DEA [satellite datasets](02_DEA.ipynb)\n", " * Inspecting available [DEA products and measurements](03_Products_and_measurements.ipynb)\n", " " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Background\n", "Loading data from the [Digital Earth Australia (DEA)](https://www.ga.gov.au/dea) instance of the [Open Data Cube](https://www.opendatacube.org/) requires the construction of a data query that specifies the what, where, and when of the data request.\n", "Each query returns a [multi-dimensional xarray object](http://xarray.pydata.org/en/stable/) containing the contents of your query.\n", "It is essential to understand the `xarray` data structures as they are fundamental to the structure of data loaded from the datacube.\n", "Manipulations, transformations and visualisation of `xarray` objects provide datacube users with the ability to explore and analyse DEA datasets, as well as pose and answer scientific questions." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Description\n", "This notebook will introduce how to load data from the DEA datacube through the construction of a query and use of the `dc.load()` function.\n", "Topics covered include:\n", "\n", "1. Loading data using `dc.load()`\n", "2. Interpreting the resulting `xarray.Dataset` object\n", " * Inspecting an individual `xarray.DataArray`\n", "3. Customising parameters passed to the `dc.load()` function\n", " * Loading specific measurements\n", " * Loading data for coordinates in a custom coordinate reference system (CRS)\n", " * Projecting data to a new CRS and spatial resolution \n", " * Specifying a specific spatial resampling method\n", "4. Loading data using a reusable dictionary query\n", "5. Loading matching data from multiple products using `like`\n", "6. Adding a progress bar to the data load\n", "\n", "***" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Getting started\n", "To run this introduction to loading data from DEA, run all the cells in the notebook starting with the \"Load packages\" cell. For help with running notebook cells, refer back to the [Jupyter Notebooks notebook](01_Jupyter_notebooks.ipynb)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Load packages\n", "The `datacube` package is required to query the datacube database and load some data. \n", "The `with_ui_cbk` function from `odc.ui` enables a progress bar when loading large amounts of data." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import datacube\n", "from odc.ui import with_ui_cbk" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Connect to the datacube\n", "The next step is to connect to the datacube database.\n", "The resulting `dc` datacube object can then be used to load data.\n", "The `app` parameter is a unique name used to identify the notebook that does not have any effect on the analysis." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "dc = datacube.Datacube(app=\"04_Loading_data\")" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Loading data using `dc.load()`\n", "\n", "Loading data from the datacube uses the [dc.load()](https://datacube-core.readthedocs.io/en/latest/api/indexed-data/generate/datacube.Datacube.load.html) function.\n", "\n", "The function requires the following minimum arguments:\n", "\n", "* `product`: The data product to load (to revise DEA products, see the [Products and measurements](03_Products_and_measurements.ipynb) notebook).\n", "* `x`: The spatial region in the *x* dimension. By default, the *x* and *y* arguments accept queries in a geographical co-ordinate system WGS84, identified by the EPSG code *4326*.\n", "* `y`: The spatial region in the *y* dimension. The dimensions ``longitude``/``latitude`` and ``x``/``y`` can be used interchangeably.\n", "* `time`: The temporal extent. The time dimension can be specified using a tuple of datetime objects or strings in the \"YYYY\", \"YYYY-MM\" or \"YYYY-MM-DD\" format. \n", "\n", "For example, to load 2015 data from the [Landsat 8 NBAR-T annual geomedian product](https://explorer.dea.ga.gov.au/ga_ls8c_nbart_gm_cyear_3) for Moreton Bay in southern Queensland, use the following parameters:\n", "\n", "* `product`: `ga_ls8c_nbart_gm_cyear_3`\n", "* `x`: `(153.3, 153.4)`\n", "* `y`: `(-27.5, -27.6)`\n", "* `time`: `(\"2015-01-01\", \"2015-12-31\")`\n", "\n", "Run the following cell to load all datasets from the `ga_ls8c_nbart_gm_cyear_3` product that match this spatial and temporal extent:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.Dataset>\n",
       "Dimensions:      (time: 1, y: 424, x: 384)\n",
       "Coordinates:\n",
       "  * time         (time) datetime64[ns] 2015-07-02T11:59:59.999999\n",
       "  * y            (y) float64 -3.156e+06 -3.156e+06 ... -3.168e+06 -3.168e+06\n",
       "  * x            (x) float64 2.067e+06 2.067e+06 ... 2.079e+06 2.079e+06\n",
       "    spatial_ref  int32 3577\n",
       "Data variables:\n",
       "    blue         (time, y, x) int16 469 470 474 480 481 ... 321 311 277 258 267\n",
       "    green        (time, y, x) int16 510 513 517 524 527 ... 471 490 434 364 363\n",
       "    red          (time, y, x) int16 231 235 238 241 242 ... 382 375 332 311 319\n",
       "    nir          (time, y, x) int16 94 94 95 96 97 ... 2495 2709 2466 2136 2218\n",
       "    swir1        (time, y, x) int16 56 57 56 57 58 ... 1388 1428 1181 1018 1050\n",
       "    swir2        (time, y, x) int16 46 46 47 48 47 48 ... 715 710 580 491 491\n",
       "    sdev         (time, y, x) float32 0.003447 0.003274 ... 0.001281 0.0019\n",
       "    edev         (time, y, x) float32 132.5 127.3 125.9 ... 205.5 175.0 164.0\n",
       "    bcdev        (time, y, x) float32 0.09468 0.09092 ... 0.04037 0.03832\n",
       "    count        (time, y, x) int16 16 16 16 16 16 16 16 ... 13 13 12 12 12 12\n",
       "Attributes:\n",
       "    crs:           EPSG:3577\n",
       "    grid_mapping:  spatial_ref
" ], "text/plain": [ "\n", "Dimensions: (time: 1, y: 424, x: 384)\n", "Coordinates:\n", " * time (time) datetime64[ns] 2015-07-02T11:59:59.999999\n", " * y (y) float64 -3.156e+06 -3.156e+06 ... -3.168e+06 -3.168e+06\n", " * x (x) float64 2.067e+06 2.067e+06 ... 2.079e+06 2.079e+06\n", " spatial_ref int32 3577\n", "Data variables:\n", " blue (time, y, x) int16 469 470 474 480 481 ... 321 311 277 258 267\n", " green (time, y, x) int16 510 513 517 524 527 ... 471 490 434 364 363\n", " red (time, y, x) int16 231 235 238 241 242 ... 382 375 332 311 319\n", " nir (time, y, x) int16 94 94 95 96 97 ... 2495 2709 2466 2136 2218\n", " swir1 (time, y, x) int16 56 57 56 57 58 ... 1388 1428 1181 1018 1050\n", " swir2 (time, y, x) int16 46 46 47 48 47 48 ... 715 710 580 491 491\n", " sdev (time, y, x) float32 0.003447 0.003274 ... 0.001281 0.0019\n", " edev (time, y, x) float32 132.5 127.3 125.9 ... 205.5 175.0 164.0\n", " bcdev (time, y, x) float32 0.09468 0.09092 ... 0.04037 0.03832\n", " count (time, y, x) int16 16 16 16 16 16 16 16 ... 13 13 12 12 12 12\n", "Attributes:\n", " crs: EPSG:3577\n", " grid_mapping: spatial_ref" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ds = dc.load(product=\"ga_ls8c_nbart_gm_cyear_3\",\n", " x=(153.3, 153.4),\n", " y=(-27.5, -27.6),\n", " time=(\"2015-01-01\", \"2015-12-31\"))\n", "\n", "ds" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Interpreting the resulting `xarray.Dataset`\n", "The variable `ds` has returned an `xarray.Dataset` containing all data that matched the spatial and temporal query parameters inputted into `dc.load`.\n", "\n", "*Dimensions* \n", "\n", "* This header identifies the number of timesteps returned in the search (`time: 1`) as well as the number of pixels in the `x` and `y` directions of the data query.\n", "\n", "*Coordinates* \n", "\n", "* `time` identifies the date attributed to each returned timestep.\n", "* `x` and `y` are the coordinates for each pixel within the spatial bounds of the query.\n", "\n", "*Data variables*\n", "\n", "* These are the measurements available for the nominated product. \n", "For every date (`time`) returned by the query, the measured value at each pixel (`y`, `x`) is returned as an array for each measurement.\n", "Each data variable is itself an `xarray.DataArray` object ([see below](#Inspecting-an-individual-xarray.DataArray)). \n", "\n", "*Attributes*\n", "\n", "* `crs` identifies the coordinate reference system (CRS) of the loaded data. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Inspecting an individual `xarray.DataArray`\n", "The `xarray.Dataset` loaded above is itself a collection of individual `xarray.DataArray` objects that hold the actual data for each data variable/measurement. \n", "For example, all measurements listed under _Data variables_ above (e.g. `blue`, `green`, `red`, `nir`, `swir1`, `swir2`) are `xarray.DataArray` objects.\n", "\n", "These `xarray.DataArray` objects can be inspected or interacted with by using either of the following syntaxes:\n", "```\n", "ds[\"measurement_name\"]\n", "```\n", "or\n", "```\n", "ds.measurement_name\n", "```\n", "\n", "The ability to access individual variables means that these can be directly viewed, or further manipulated to create new variables. \n", "For example, run the following cell to access data from the near infra-red satellite band (i.e. `nir`):" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.DataArray 'nir' (time: 1, y: 424, x: 384)>\n",
       "array([[[  94,   94,   95, ...,   85,   89,   93],\n",
       "        [  94,   98,   92, ...,   85,   86,  121],\n",
       "        [  93,   95,   91, ...,   94,  103,  141],\n",
       "        ...,\n",
       "        [3155, 2848, 2651, ..., 2060, 2365, 2472],\n",
       "        [2749, 2921, 2699, ..., 2081, 2270, 2477],\n",
       "        [2535, 2865, 2649, ..., 2466, 2136, 2218]]], dtype=int16)\n",
       "Coordinates:\n",
       "  * time         (time) datetime64[ns] 2015-07-02T11:59:59.999999\n",
       "  * y            (y) float64 -3.156e+06 -3.156e+06 ... -3.168e+06 -3.168e+06\n",
       "  * x            (x) float64 2.067e+06 2.067e+06 ... 2.079e+06 2.079e+06\n",
       "    spatial_ref  int32 3577\n",
       "Attributes:\n",
       "    units:         1\n",
       "    nodata:        -999\n",
       "    crs:           EPSG:3577\n",
       "    grid_mapping:  spatial_ref
" ], "text/plain": [ "\n", "array([[[ 94, 94, 95, ..., 85, 89, 93],\n", " [ 94, 98, 92, ..., 85, 86, 121],\n", " [ 93, 95, 91, ..., 94, 103, 141],\n", " ...,\n", " [3155, 2848, 2651, ..., 2060, 2365, 2472],\n", " [2749, 2921, 2699, ..., 2081, 2270, 2477],\n", " [2535, 2865, 2649, ..., 2466, 2136, 2218]]], dtype=int16)\n", "Coordinates:\n", " * time (time) datetime64[ns] 2015-07-02T11:59:59.999999\n", " * y (y) float64 -3.156e+06 -3.156e+06 ... -3.168e+06 -3.168e+06\n", " * x (x) float64 2.067e+06 2.067e+06 ... 2.079e+06 2.079e+06\n", " spatial_ref int32 3577\n", "Attributes:\n", " units: 1\n", " nodata: -999\n", " crs: EPSG:3577\n", " grid_mapping: spatial_ref" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ds.nir" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that the object header informs us that it is an `xarray.DataArray` containing data for the `nir` satellite band. \n", "\n", "Like an `xarray.Dataset`, the array also includes information about the data's **dimensions** (i.e. `(time: 1, y: 508, x: 461)`), **coordinates** and **attributes**.\n", "This particular data variable/measurement contains some additional information that is specific to the `nir` band, including details of array's nodata value (i.e. `nodata: -999`).\n", "\n", "> For a more in-depth introduction to `xarray` data structures, refer to the [official xarray documentation](http://xarray.pydata.org/en/stable/data-structures.html)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Customising the `dc.load()` function\n", "\n", "The `dc.load()` function can be tailored to refine a query.\n", "\n", "Customisation options include:\n", "\n", "* `measurements:` This argument is used to provide a list of measurement names to load, as listed in `dc.list_measurements()`. \n", "For satellite datasets, measurements contain data for each individual satellite band (e.g. near infrared). \n", "If not provided, all measurements for the product will be returned.\n", "* `crs:` The coordinate reference system (CRS) of the query's `x` and `y` coordinates is assumed to be `WGS84`/`EPSG:4326` unless the `crs` field is supplied, even if the stored data is in another projection or the `output_crs` is specified. \n", "The `crs` parameter is required if the query's coordinates are in any other CRS.\n", "* `group_by:` Satellite datasets based around scenes can have multiple observations per day with slightly different time stamps as the satellite collects data along its path.\n", "These observations can be combined by reducing the `time` dimension to the day level using `group_by=solar_day`.\n", "* `output_crs` and `resolution`: To reproject or change the resolution the data, supply the `output_crs` and `resolution` fields. \n", "* `resampling`: This argument allows you to specify a custom spatial resampling method to use when data is reprojected into a different CRS. \n", "\n", "Example syntax on the use of these options follows in the cells below.\n", "\n", "> For help or more customisation options, run `help(dc.load)` in an empty cell or visit the function's [documentation page](https://datacube-core.readthedocs.io/en/latest/api/indexed-data/generate/datacube.Datacube.load.html)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Specifying measurements\n", "By default, `dc.load()` will load *all* measurements in a product.\n", "\n", "To load data from the `red`, `green` and `blue` satellite bands only, add `measurements=[\"red\", \"green\", \"blue\"]` to the query:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.Dataset>\n",
       "Dimensions:      (time: 1, y: 424, x: 384)\n",
       "Coordinates:\n",
       "  * time         (time) datetime64[ns] 2015-07-02T11:59:59.999999\n",
       "  * y            (y) float64 -3.156e+06 -3.156e+06 ... -3.168e+06 -3.168e+06\n",
       "  * x            (x) float64 2.067e+06 2.067e+06 ... 2.079e+06 2.079e+06\n",
       "    spatial_ref  int32 3577\n",
       "Data variables:\n",
       "    red          (time, y, x) int16 231 235 238 241 242 ... 382 375 332 311 319\n",
       "    green        (time, y, x) int16 510 513 517 524 527 ... 471 490 434 364 363\n",
       "    blue         (time, y, x) int16 469 470 474 480 481 ... 321 311 277 258 267\n",
       "Attributes:\n",
       "    crs:           EPSG:3577\n",
       "    grid_mapping:  spatial_ref
" ], "text/plain": [ "\n", "Dimensions: (time: 1, y: 424, x: 384)\n", "Coordinates:\n", " * time (time) datetime64[ns] 2015-07-02T11:59:59.999999\n", " * y (y) float64 -3.156e+06 -3.156e+06 ... -3.168e+06 -3.168e+06\n", " * x (x) float64 2.067e+06 2.067e+06 ... 2.079e+06 2.079e+06\n", " spatial_ref int32 3577\n", "Data variables:\n", " red (time, y, x) int16 231 235 238 241 242 ... 382 375 332 311 319\n", " green (time, y, x) int16 510 513 517 524 527 ... 471 490 434 364 363\n", " blue (time, y, x) int16 469 470 474 480 481 ... 321 311 277 258 267\n", "Attributes:\n", " crs: EPSG:3577\n", " grid_mapping: spatial_ref" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Note the optional inclusion of the measurements list\n", "ds_rgb = dc.load(product=\"ga_ls8c_nbart_gm_cyear_3\",\n", " measurements=[\"red\", \"green\", \"blue\"],\n", " x=(153.3, 153.4),\n", " y=(-27.5, -27.6),\n", " time=(\"2015-01-01\", \"2015-12-31\"))\n", "\n", "ds_rgb" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that the **Data variables** component of the `xarray.Dataset` now includes only the measurements specified in the query (i.e. the `red`, `green` and `blue` satellite bands)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Loading data for coordinates in any CRS\n", "By default, `dc.load()` assumes that the queried `x` and `y` coordinates are in the `WGS84`/`EPSG:4326` CRS.\n", "If these coordinates are in a different coordinate system, specify this using the `crs` parameter.\n", "\n", "The example cell below loads data for a set of `x` and `y` coordinates defined in Australian Albers (`EPSG:3577`), ensuring that the `dc.load()` function accounts for this by including `crs=\"EPSG:3577\"`:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.Dataset>\n",
       "Dimensions:      (time: 1, y: 423, x: 259)\n",
       "Coordinates:\n",
       "  * time         (time) datetime64[ns] 2015-07-02T11:59:59.999999\n",
       "  * y            (y) float64 -3.156e+06 -3.156e+06 ... -3.168e+06 -3.168e+06\n",
       "  * x            (x) float64 2.069e+06 2.069e+06 ... 2.077e+06 2.077e+06\n",
       "    spatial_ref  int32 3577\n",
       "Data variables:\n",
       "    blue         (time, y, x) int16 462 462 459 456 458 ... 424 444 441 399 378\n",
       "    green        (time, y, x) int16 476 475 470 469 470 ... 481 491 484 450 437\n",
       "    red          (time, y, x) int16 213 211 208 208 209 ... 249 251 245 222 213\n",
       "    nir          (time, y, x) int16 82 83 80 78 78 79 ... 99 109 115 112 87 78\n",
       "    swir1        (time, y, x) int16 50 48 48 47 46 47 46 ... 56 65 65 63 46 38\n",
       "    swir2        (time, y, x) int16 41 40 39 37 36 38 37 ... 45 52 51 49 35 30\n",
       "    sdev         (time, y, x) float32 0.003467 0.00352 ... 0.005748 0.005211\n",
       "    edev         (time, y, x) float32 94.9 94.76 87.63 ... 183.1 109.6 116.1\n",
       "    bcdev        (time, y, x) float32 0.07639 0.07612 0.06811 ... 0.08707 0.0849\n",
       "    count        (time, y, x) int16 17 17 17 17 17 17 17 ... 16 17 16 16 14 14\n",
       "Attributes:\n",
       "    crs:           EPSG:3577\n",
       "    grid_mapping:  spatial_ref
" ], "text/plain": [ "\n", "Dimensions: (time: 1, y: 423, x: 259)\n", "Coordinates:\n", " * time (time) datetime64[ns] 2015-07-02T11:59:59.999999\n", " * y (y) float64 -3.156e+06 -3.156e+06 ... -3.168e+06 -3.168e+06\n", " * x (x) float64 2.069e+06 2.069e+06 ... 2.077e+06 2.077e+06\n", " spatial_ref int32 3577\n", "Data variables:\n", " blue (time, y, x) int16 462 462 459 456 458 ... 424 444 441 399 378\n", " green (time, y, x) int16 476 475 470 469 470 ... 481 491 484 450 437\n", " red (time, y, x) int16 213 211 208 208 209 ... 249 251 245 222 213\n", " nir (time, y, x) int16 82 83 80 78 78 79 ... 99 109 115 112 87 78\n", " swir1 (time, y, x) int16 50 48 48 47 46 47 46 ... 56 65 65 63 46 38\n", " swir2 (time, y, x) int16 41 40 39 37 36 38 37 ... 45 52 51 49 35 30\n", " sdev (time, y, x) float32 0.003467 0.00352 ... 0.005748 0.005211\n", " edev (time, y, x) float32 94.9 94.76 87.63 ... 183.1 109.6 116.1\n", " bcdev (time, y, x) float32 0.07639 0.07612 0.06811 ... 0.08707 0.0849\n", " count (time, y, x) int16 17 17 17 17 17 17 17 ... 16 17 16 16 14 14\n", "Attributes:\n", " crs: EPSG:3577\n", " grid_mapping: spatial_ref" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Note the new `x` and `y` coordinates and `crs` parameter\n", "ds_custom_crs = dc.load(product=\"ga_ls8c_nbart_gm_cyear_3\",\n", " time=(\"2015-01-01\", \"2015-12-31\"),\n", " x=(2069310, 2077064),\n", " y=(-3155823, -3168513),\n", " crs=\"EPSG:3577\")\n", "\n", "ds_custom_crs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### CRS reprojection\n", "Certain applications may require that data is output into a specific CRS.\n", "Data can be reprojected by specifying the new `output_crs` and identifying the `resolution` required.\n", "\n", "The example cell below reprojects data to a new CRS (UTM Zone 56S, `EPSG:32756`) and resolution (250 x 250 m). Note that for most CRSs, the first resolution value is negative (e.g. `(-250, 250)`):" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.Dataset>\n",
       "Dimensions:      (time: 1, y: 45, x: 40)\n",
       "Coordinates:\n",
       "  * time         (time) datetime64[ns] 2015-07-02T11:59:59.999999\n",
       "  * y            (y) float64 6.958e+06 6.958e+06 ... 6.947e+06 6.947e+06\n",
       "  * x            (x) float64 5.296e+05 5.299e+05 ... 5.391e+05 5.394e+05\n",
       "    spatial_ref  int32 32756\n",
       "Data variables:\n",
       "    blue         (time, y, x) int16 456 447 440 432 424 ... 422 429 420 407 423\n",
       "    green        (time, y, x) int16 464 444 429 418 401 ... 493 491 479 466 476\n",
       "    red          (time, y, x) int16 207 195 187 179 170 ... 271 252 241 240 243\n",
       "    nir          (time, y, x) int16 81 78 79 75 75 77 75 ... 86 86 88 86 93 107\n",
       "    swir1        (time, y, x) int16 48 46 46 43 42 42 43 ... 43 45 47 45 51 65\n",
       "    swir2        (time, y, x) int16 40 37 38 35 34 35 35 ... 35 37 38 36 42 54\n",
       "    sdev         (time, y, x) float32 0.003807 0.003953 ... 0.005075 0.01121\n",
       "    edev         (time, y, x) float32 98.53 98.11 96.56 ... 117.3 124.9 166.8\n",
       "    bcdev        (time, y, x) float32 0.07894 0.07555 0.07484 ... 0.1178 0.1544\n",
       "    count        (time, y, x) int16 17 17 17 17 16 16 16 ... 14 15 15 15 15 15\n",
       "Attributes:\n",
       "    crs:           EPSG:32756\n",
       "    grid_mapping:  spatial_ref
" ], "text/plain": [ "\n", "Dimensions: (time: 1, y: 45, x: 40)\n", "Coordinates:\n", " * time (time) datetime64[ns] 2015-07-02T11:59:59.999999\n", " * y (y) float64 6.958e+06 6.958e+06 ... 6.947e+06 6.947e+06\n", " * x (x) float64 5.296e+05 5.299e+05 ... 5.391e+05 5.394e+05\n", " spatial_ref int32 32756\n", "Data variables:\n", " blue (time, y, x) int16 456 447 440 432 424 ... 422 429 420 407 423\n", " green (time, y, x) int16 464 444 429 418 401 ... 493 491 479 466 476\n", " red (time, y, x) int16 207 195 187 179 170 ... 271 252 241 240 243\n", " nir (time, y, x) int16 81 78 79 75 75 77 75 ... 86 86 88 86 93 107\n", " swir1 (time, y, x) int16 48 46 46 43 42 42 43 ... 43 45 47 45 51 65\n", " swir2 (time, y, x) int16 40 37 38 35 34 35 35 ... 35 37 38 36 42 54\n", " sdev (time, y, x) float32 0.003807 0.003953 ... 0.005075 0.01121\n", " edev (time, y, x) float32 98.53 98.11 96.56 ... 117.3 124.9 166.8\n", " bcdev (time, y, x) float32 0.07894 0.07555 0.07484 ... 0.1178 0.1544\n", " count (time, y, x) int16 17 17 17 17 16 16 16 ... 14 15 15 15 15 15\n", "Attributes:\n", " crs: EPSG:32756\n", " grid_mapping: spatial_ref" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ds_reprojected = dc.load(product=\"ga_ls8c_nbart_gm_cyear_3\",\n", " x=(153.3, 153.4),\n", " y=(-27.5, -27.6),\n", " time=(\"2015-01-01\", \"2015-12-31\"),\n", " output_crs=\"EPSG:32756\",\n", " resolution=(-250, 250))\n", "\n", "ds_reprojected" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that the `crs` attribute in the **Attributes** section has changed to `EPSG:32756`. \n", "Due to the larger 250 m resolution, there are now fewer pixels on the `x` and `y` dimensions (e.g. `x: 40, y: 45` compared to `x: 461, y: 508` in earlier examples).\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Spatial resampling methods\n", "When a product is re-projected to a different CRS and/or resolution, the new pixel grid may differ from the original input pixels by size, number and alignment.\n", "It is therefore necessary to apply a spatial \"resampling\" rule that allocates input pixel values into the new pixel grid.\n", "\n", "By default, `dc.load()` resamples pixel values using \"nearest neighbour\" resampling, which allocates each new pixel with the value of the closest input pixel.\n", "Depending on the type of data and the analysis being run, this may not be the most appropriate choice (e.g. for continuous data).\n", "\n", "The `resampling` parameter in `dc.load()` allows you to choose a custom resampling method from the following options: \n", "\n", "```\n", "\"nearest\", \"cubic\", \"bilinear\", \"cubic_spline\", \"lanczos\", \n", "\"average\", \"mode\", \"gauss\", \"max\", \"min\", \"med\", \"q1\", \"q3\"\n", "```\n", "\n", "The example cell below requests that all loaded data is resampled using \"average\" resampling:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.Dataset>\n",
       "Dimensions:      (time: 1, y: 51, x: 47)\n",
       "Coordinates:\n",
       "  * time         (time) datetime64[ns] 2015-07-02T11:59:59.999999\n",
       "  * y            (y) float64 -3.156e+06 -3.156e+06 ... -3.168e+06 -3.168e+06\n",
       "  * x            (x) float64 2.067e+06 2.068e+06 ... 2.079e+06 2.079e+06\n",
       "    spatial_ref  int32 3577\n",
       "Data variables:\n",
       "    blue         (time, y, x) int16 483 473 478 481 492 ... 471 573 371 321 285\n",
       "    green        (time, y, x) int16 524 516 528 534 547 ... 567 704 443 490 501\n",
       "    red          (time, y, x) int16 240 235 245 256 264 ... 446 616 261 412 367\n",
       "    nir          (time, y, x) int16 100 91 87 88 90 87 ... 284 412 194 1869 2896\n",
       "    swir1        (time, y, x) int16 60 53 49 51 53 52 ... 46 130 203 96 926 1260\n",
       "    swir2        (time, y, x) int16 49 44 40 41 42 42 ... 35 93 131 59 464 584\n",
       "    sdev         (time, y, x) float32 0.003118 0.00266 ... 0.00298 0.0007673\n",
       "    edev         (time, y, x) float32 129.6 118.5 118.7 ... 160.0 181.4 189.9\n",
       "    bcdev        (time, y, x) float32 0.09975 0.09054 0.0885 ... 0.05167 0.03022\n",
       "    count        (time, y, x) int16 17 17 16 15 16 16 17 ... 14 14 14 13 12 12\n",
       "Attributes:\n",
       "    crs:           EPSG:3577\n",
       "    grid_mapping:  spatial_ref
" ], "text/plain": [ "\n", "Dimensions: (time: 1, y: 51, x: 47)\n", "Coordinates:\n", " * time (time) datetime64[ns] 2015-07-02T11:59:59.999999\n", " * y (y) float64 -3.156e+06 -3.156e+06 ... -3.168e+06 -3.168e+06\n", " * x (x) float64 2.067e+06 2.068e+06 ... 2.079e+06 2.079e+06\n", " spatial_ref int32 3577\n", "Data variables:\n", " blue (time, y, x) int16 483 473 478 481 492 ... 471 573 371 321 285\n", " green (time, y, x) int16 524 516 528 534 547 ... 567 704 443 490 501\n", " red (time, y, x) int16 240 235 245 256 264 ... 446 616 261 412 367\n", " nir (time, y, x) int16 100 91 87 88 90 87 ... 284 412 194 1869 2896\n", " swir1 (time, y, x) int16 60 53 49 51 53 52 ... 46 130 203 96 926 1260\n", " swir2 (time, y, x) int16 49 44 40 41 42 42 ... 35 93 131 59 464 584\n", " sdev (time, y, x) float32 0.003118 0.00266 ... 0.00298 0.0007673\n", " edev (time, y, x) float32 129.6 118.5 118.7 ... 160.0 181.4 189.9\n", " bcdev (time, y, x) float32 0.09975 0.09054 0.0885 ... 0.05167 0.03022\n", " count (time, y, x) int16 17 17 16 15 16 16 17 ... 14 14 14 13 12 12\n", "Attributes:\n", " crs: EPSG:3577\n", " grid_mapping: spatial_ref" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Note the additional `resampling` parameter\n", "ds_averesampling = dc.load(product=\"ga_ls8c_nbart_gm_cyear_3\",\n", " x=(153.3, 153.4),\n", " y=(-27.5, -27.6),\n", " time=(\"2015-01-01\", \"2015-12-31\"),\n", " resolution=(-250, 250),\n", " resampling=\"average\")\n", "\n", "ds_averesampling\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Python dictionaries can be used to request different sampling methods for different measurements. \n", "This can be particularly useful when some measurements contain contain categorical data which require resampling methods such as \"nearest\" or \"mode\" that do not modify the input pixel values.\n", "\n", "The example cell below specifies `resampling={\"red\": \"nearest\", \"*\": \"average\"}`, which implements \"nearest\" neighbour resampling for the `red` satellite band only. `\"*\": \"average\"` will apply \"average\" resampling for all other satellite bands:\n" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.Dataset>\n",
       "Dimensions:      (time: 1, y: 45, x: 40)\n",
       "Coordinates:\n",
       "  * time         (time) datetime64[ns] 2015-07-02T11:59:59.999999\n",
       "  * y            (y) float64 6.958e+06 6.958e+06 ... 6.947e+06 6.947e+06\n",
       "  * x            (x) float64 5.296e+05 5.299e+05 ... 5.391e+05 5.394e+05\n",
       "    spatial_ref  int32 32756\n",
       "Data variables:\n",
       "    blue         (time, y, x) int16 458 450 439 433 424 ... 426 424 420 408 416\n",
       "    green        (time, y, x) int16 469 451 428 419 402 ... 505 487 480 467 471\n",
       "    red          (time, y, x) int16 207 195 187 179 170 ... 271 252 241 240 243\n",
       "    nir          (time, y, x) int16 81 79 78 76 75 77 75 ... 84 91 88 86 91 102\n",
       "    swir1        (time, y, x) int16 48 46 46 44 42 42 43 ... 42 47 46 45 49 60\n",
       "    swir2        (time, y, x) int16 40 38 38 36 34 35 35 ... 34 38 38 36 40 49\n",
       "    sdev         (time, y, x) float32 0.003786 0.003991 ... 0.004875 0.0102\n",
       "    edev         (time, y, x) float32 99.68 99.1 99.31 ... 116.5 124.3 160.1\n",
       "    bcdev        (time, y, x) float32 0.07825 0.07446 0.07933 ... 0.1138 0.1466\n",
       "    count        (time, y, x) int16 17 17 17 17 16 16 16 ... 14 15 15 15 15 15\n",
       "Attributes:\n",
       "    crs:           EPSG:32756\n",
       "    grid_mapping:  spatial_ref
" ], "text/plain": [ "\n", "Dimensions: (time: 1, y: 45, x: 40)\n", "Coordinates:\n", " * time (time) datetime64[ns] 2015-07-02T11:59:59.999999\n", " * y (y) float64 6.958e+06 6.958e+06 ... 6.947e+06 6.947e+06\n", " * x (x) float64 5.296e+05 5.299e+05 ... 5.391e+05 5.394e+05\n", " spatial_ref int32 32756\n", "Data variables:\n", " blue (time, y, x) int16 458 450 439 433 424 ... 426 424 420 408 416\n", " green (time, y, x) int16 469 451 428 419 402 ... 505 487 480 467 471\n", " red (time, y, x) int16 207 195 187 179 170 ... 271 252 241 240 243\n", " nir (time, y, x) int16 81 79 78 76 75 77 75 ... 84 91 88 86 91 102\n", " swir1 (time, y, x) int16 48 46 46 44 42 42 43 ... 42 47 46 45 49 60\n", " swir2 (time, y, x) int16 40 38 38 36 34 35 35 ... 34 38 38 36 40 49\n", " sdev (time, y, x) float32 0.003786 0.003991 ... 0.004875 0.0102\n", " edev (time, y, x) float32 99.68 99.1 99.31 ... 116.5 124.3 160.1\n", " bcdev (time, y, x) float32 0.07825 0.07446 0.07933 ... 0.1138 0.1466\n", " count (time, y, x) int16 17 17 17 17 16 16 16 ... 14 15 15 15 15 15\n", "Attributes:\n", " crs: EPSG:32756\n", " grid_mapping: spatial_ref" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ds_customresampling = dc.load(product=\"ga_ls8c_nbart_gm_cyear_3\",\n", " x=(153.3, 153.4),\n", " y=(-27.5, -27.6),\n", " time=(\"2015-01-01\", \"2015-12-31\"),\n", " output_crs=\"EPSG:32756\",\n", " resolution=(-250, 250),\n", " resampling={\"red\": \"nearest\", \"*\": \"average\"})\n", "\n", "ds_customresampling" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "> For more information about spatial resampling methods, see the [following guide](https://rasterio.readthedocs.io/en/stable/topics/resampling.html)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Loading data using the query dictionary syntax\n", "It is often useful to re-use a set of query parameters to load data from multiple products.\n", "To achieve this, load data using the \"query dictionary\" syntax.\n", "This involves placing the query parameters inside a Python dictionary object which can be re-used for multiple data loads:" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "query = {\"x\": (153.3, 153.4),\n", " \"y\": (-27.5, -27.6),\n", " \"time\": (\"2015-01-01\", \"2015-12-31\")}\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The query dictionary object can be added as an input to `dc.load()`. \n", "\n", "> The `**` syntax below is Python's \"keyword argument unpacking\" operator.\n", "This operator takes the named query parameters listed in the query dictionary (e.g. `\"x\": (153.3, 153.4)`), and \"unpacks\" them into the `dc.load()` function as new arguments. \n", "For more information about unpacking operators, refer to the [Python documentation](https://docs.python.org/3/tutorial/controlflow.html#unpacking-argument-lists)" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.Dataset>\n",
       "Dimensions:      (time: 1, y: 424, x: 384)\n",
       "Coordinates:\n",
       "  * time         (time) datetime64[ns] 2015-07-02T11:59:59.999999\n",
       "  * y            (y) float64 -3.156e+06 -3.156e+06 ... -3.168e+06 -3.168e+06\n",
       "  * x            (x) float64 2.067e+06 2.067e+06 ... 2.079e+06 2.079e+06\n",
       "    spatial_ref  int32 3577\n",
       "Data variables:\n",
       "    blue         (time, y, x) int16 469 470 474 480 481 ... 321 311 277 258 267\n",
       "    green        (time, y, x) int16 510 513 517 524 527 ... 471 490 434 364 363\n",
       "    red          (time, y, x) int16 231 235 238 241 242 ... 382 375 332 311 319\n",
       "    nir          (time, y, x) int16 94 94 95 96 97 ... 2495 2709 2466 2136 2218\n",
       "    swir1        (time, y, x) int16 56 57 56 57 58 ... 1388 1428 1181 1018 1050\n",
       "    swir2        (time, y, x) int16 46 46 47 48 47 48 ... 715 710 580 491 491\n",
       "    sdev         (time, y, x) float32 0.003447 0.003274 ... 0.001281 0.0019\n",
       "    edev         (time, y, x) float32 132.5 127.3 125.9 ... 205.5 175.0 164.0\n",
       "    bcdev        (time, y, x) float32 0.09468 0.09092 ... 0.04037 0.03832\n",
       "    count        (time, y, x) int16 16 16 16 16 16 16 16 ... 13 13 12 12 12 12\n",
       "Attributes:\n",
       "    crs:           EPSG:3577\n",
       "    grid_mapping:  spatial_ref
" ], "text/plain": [ "\n", "Dimensions: (time: 1, y: 424, x: 384)\n", "Coordinates:\n", " * time (time) datetime64[ns] 2015-07-02T11:59:59.999999\n", " * y (y) float64 -3.156e+06 -3.156e+06 ... -3.168e+06 -3.168e+06\n", " * x (x) float64 2.067e+06 2.067e+06 ... 2.079e+06 2.079e+06\n", " spatial_ref int32 3577\n", "Data variables:\n", " blue (time, y, x) int16 469 470 474 480 481 ... 321 311 277 258 267\n", " green (time, y, x) int16 510 513 517 524 527 ... 471 490 434 364 363\n", " red (time, y, x) int16 231 235 238 241 242 ... 382 375 332 311 319\n", " nir (time, y, x) int16 94 94 95 96 97 ... 2495 2709 2466 2136 2218\n", " swir1 (time, y, x) int16 56 57 56 57 58 ... 1388 1428 1181 1018 1050\n", " swir2 (time, y, x) int16 46 46 47 48 47 48 ... 715 710 580 491 491\n", " sdev (time, y, x) float32 0.003447 0.003274 ... 0.001281 0.0019\n", " edev (time, y, x) float32 132.5 127.3 125.9 ... 205.5 175.0 164.0\n", " bcdev (time, y, x) float32 0.09468 0.09092 ... 0.04037 0.03832\n", " count (time, y, x) int16 16 16 16 16 16 16 16 ... 13 13 12 12 12 12\n", "Attributes:\n", " crs: EPSG:3577\n", " grid_mapping: spatial_ref" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ds = dc.load(product=\"ga_ls8c_nbart_gm_cyear_3\",\n", " **query)\n", "\n", "ds" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Query dictionaries can contain any set of parameters that would usually be provided to `dc.load()`:" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.Dataset>\n",
       "Dimensions:      (time: 1, y: 45, x: 40)\n",
       "Coordinates:\n",
       "  * time         (time) datetime64[ns] 2015-07-02T11:59:59.999999\n",
       "  * y            (y) float64 6.958e+06 6.958e+06 ... 6.947e+06 6.947e+06\n",
       "  * x            (x) float64 5.296e+05 5.299e+05 ... 5.391e+05 5.394e+05\n",
       "    spatial_ref  int32 32756\n",
       "Data variables:\n",
       "    blue         (time, y, x) int16 456 447 440 432 424 ... 422 429 420 407 423\n",
       "    green        (time, y, x) int16 464 444 429 418 401 ... 493 491 479 466 476\n",
       "    red          (time, y, x) int16 207 195 187 179 170 ... 271 252 241 240 243\n",
       "    nir          (time, y, x) int16 81 78 79 75 75 77 75 ... 86 86 88 86 93 107\n",
       "    swir1        (time, y, x) int16 48 46 46 43 42 42 43 ... 43 45 47 45 51 65\n",
       "    swir2        (time, y, x) int16 40 37 38 35 34 35 35 ... 35 37 38 36 42 54\n",
       "    sdev         (time, y, x) float32 0.003807 0.003953 ... 0.005075 0.01121\n",
       "    edev         (time, y, x) float32 98.53 98.11 96.56 ... 117.3 124.9 166.8\n",
       "    bcdev        (time, y, x) float32 0.07894 0.07555 0.07484 ... 0.1178 0.1544\n",
       "    count        (time, y, x) int16 17 17 17 17 16 16 16 ... 14 15 15 15 15 15\n",
       "Attributes:\n",
       "    crs:           EPSG:32756\n",
       "    grid_mapping:  spatial_ref
" ], "text/plain": [ "\n", "Dimensions: (time: 1, y: 45, x: 40)\n", "Coordinates:\n", " * time (time) datetime64[ns] 2015-07-02T11:59:59.999999\n", " * y (y) float64 6.958e+06 6.958e+06 ... 6.947e+06 6.947e+06\n", " * x (x) float64 5.296e+05 5.299e+05 ... 5.391e+05 5.394e+05\n", " spatial_ref int32 32756\n", "Data variables:\n", " blue (time, y, x) int16 456 447 440 432 424 ... 422 429 420 407 423\n", " green (time, y, x) int16 464 444 429 418 401 ... 493 491 479 466 476\n", " red (time, y, x) int16 207 195 187 179 170 ... 271 252 241 240 243\n", " nir (time, y, x) int16 81 78 79 75 75 77 75 ... 86 86 88 86 93 107\n", " swir1 (time, y, x) int16 48 46 46 43 42 42 43 ... 43 45 47 45 51 65\n", " swir2 (time, y, x) int16 40 37 38 35 34 35 35 ... 35 37 38 36 42 54\n", " sdev (time, y, x) float32 0.003807 0.003953 ... 0.005075 0.01121\n", " edev (time, y, x) float32 98.53 98.11 96.56 ... 117.3 124.9 166.8\n", " bcdev (time, y, x) float32 0.07894 0.07555 0.07484 ... 0.1178 0.1544\n", " count (time, y, x) int16 17 17 17 17 16 16 16 ... 14 15 15 15 15 15\n", "Attributes:\n", " crs: EPSG:32756\n", " grid_mapping: spatial_ref" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "query = {\"x\": (153.3, 153.4),\n", " \"y\": (-27.5, -27.6),\n", " \"time\": (\"2015-01-01\", \"2015-12-31\"),\n", " \"output_crs\": \"EPSG:32756\",\n", " \"resolution\": (-250, 250)}\n", "\n", "ds_ls8 = dc.load(product=\"ga_ls8c_nbart_gm_cyear_3\",\n", " **query)\n", "\n", "ds_ls8\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "After specifying the reusable query, it can be easily used to load data from a different product.\n", "The example cell below loads Landsat 7 data for the same extent, time, output CRS and resolution as the previously loaded Landsat 8 data:" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.Dataset>\n",
       "Dimensions:      (time: 1, y: 45, x: 40)\n",
       "Coordinates:\n",
       "  * time         (time) datetime64[ns] 2015-07-02T11:59:59.999999\n",
       "  * y            (y) float64 6.958e+06 6.958e+06 ... 6.947e+06 6.947e+06\n",
       "  * x            (x) float64 5.296e+05 5.299e+05 ... 5.391e+05 5.394e+05\n",
       "    spatial_ref  int32 32756\n",
       "Data variables:\n",
       "    blue         (time, y, x) int16 450 425 388 385 394 ... 408 382 367 364 383\n",
       "    green        (time, y, x) int16 488 454 414 406 408 ... 510 480 467 466 465\n",
       "    red          (time, y, x) int16 254 233 206 203 205 ... 329 292 272 277 273\n",
       "    nir          (time, y, x) int16 157 150 134 141 142 ... 160 144 137 150 165\n",
       "    swir1        (time, y, x) int16 84 85 77 79 84 80 82 ... 111 82 80 72 81 97\n",
       "    swir2        (time, y, x) int16 71 71 64 71 70 66 68 ... 97 73 65 60 65 80\n",
       "    sdev         (time, y, x) float32 0.007338 0.009259 ... 0.01109 0.009572\n",
       "    edev         (time, y, x) float32 267.4 232.7 172.5 ... 202.2 200.9 227.4\n",
       "    bcdev        (time, y, x) float32 0.2071 0.2088 0.1745 ... 0.1806 0.2081\n",
       "    count        (time, y, x) int16 13 13 12 12 12 12 12 ... 13 11 11 11 12 12\n",
       "Attributes:\n",
       "    crs:           EPSG:32756\n",
       "    grid_mapping:  spatial_ref
" ], "text/plain": [ "\n", "Dimensions: (time: 1, y: 45, x: 40)\n", "Coordinates:\n", " * time (time) datetime64[ns] 2015-07-02T11:59:59.999999\n", " * y (y) float64 6.958e+06 6.958e+06 ... 6.947e+06 6.947e+06\n", " * x (x) float64 5.296e+05 5.299e+05 ... 5.391e+05 5.394e+05\n", " spatial_ref int32 32756\n", "Data variables:\n", " blue (time, y, x) int16 450 425 388 385 394 ... 408 382 367 364 383\n", " green (time, y, x) int16 488 454 414 406 408 ... 510 480 467 466 465\n", " red (time, y, x) int16 254 233 206 203 205 ... 329 292 272 277 273\n", " nir (time, y, x) int16 157 150 134 141 142 ... 160 144 137 150 165\n", " swir1 (time, y, x) int16 84 85 77 79 84 80 82 ... 111 82 80 72 81 97\n", " swir2 (time, y, x) int16 71 71 64 71 70 66 68 ... 97 73 65 60 65 80\n", " sdev (time, y, x) float32 0.007338 0.009259 ... 0.01109 0.009572\n", " edev (time, y, x) float32 267.4 232.7 172.5 ... 202.2 200.9 227.4\n", " bcdev (time, y, x) float32 0.2071 0.2088 0.1745 ... 0.1806 0.2081\n", " count (time, y, x) int16 13 13 12 12 12 12 12 ... 13 11 11 11 12 12\n", "Attributes:\n", " crs: EPSG:32756\n", " grid_mapping: spatial_ref" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ds_ls7 = dc.load(product=\"ga_ls7e_nbart_gm_cyear_3\",\n", " **query)\n", "\n", "ds_ls7" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Other helpful tricks\n", "### Loading data \"like\" another dataset\n", "Another option for loading matching data from multiple products is to use `dc.load()`'s `like` parameter.\n", "This will copy the spatial and temporal extent and the CRS/resolution from an existing dataset, and use these parameters to load new data from a new product.\n", "\n", "The example cell below loads another Landsat 7 dataset that exactly matches the `ds_ls8` dataset loaded earlier:\n" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.Dataset>\n",
       "Dimensions:      (time: 1, y: 45, x: 40)\n",
       "Coordinates:\n",
       "  * time         (time) datetime64[ns] 2015-07-02T11:59:59.999999\n",
       "  * y            (y) float64 6.958e+06 6.958e+06 ... 6.947e+06 6.947e+06\n",
       "  * x            (x) float64 5.296e+05 5.299e+05 ... 5.391e+05 5.394e+05\n",
       "    spatial_ref  int32 32756\n",
       "Data variables:\n",
       "    blue         (time, y, x) int16 450 425 388 385 394 ... 408 382 367 364 383\n",
       "    green        (time, y, x) int16 488 454 414 406 408 ... 510 480 467 466 465\n",
       "    red          (time, y, x) int16 254 233 206 203 205 ... 329 292 272 277 273\n",
       "    nir          (time, y, x) int16 157 150 134 141 142 ... 160 144 137 150 165\n",
       "    swir1        (time, y, x) int16 84 85 77 79 84 80 82 ... 111 82 80 72 81 97\n",
       "    swir2        (time, y, x) int16 71 71 64 71 70 66 68 ... 97 73 65 60 65 80\n",
       "    sdev         (time, y, x) float32 0.007338 0.009259 ... 0.01109 0.009572\n",
       "    edev         (time, y, x) float32 267.4 232.7 172.5 ... 202.2 200.9 227.4\n",
       "    bcdev        (time, y, x) float32 0.2071 0.2088 0.1745 ... 0.1806 0.2081\n",
       "    count        (time, y, x) int16 13 13 12 12 12 12 12 ... 13 11 11 11 12 12\n",
       "Attributes:\n",
       "    crs:           PROJCS["WGS 84 / UTM zone 56S",GEOGCS["WGS 84",DATUM["WGS_...\n",
       "    grid_mapping:  spatial_ref
" ], "text/plain": [ "\n", "Dimensions: (time: 1, y: 45, x: 40)\n", "Coordinates:\n", " * time (time) datetime64[ns] 2015-07-02T11:59:59.999999\n", " * y (y) float64 6.958e+06 6.958e+06 ... 6.947e+06 6.947e+06\n", " * x (x) float64 5.296e+05 5.299e+05 ... 5.391e+05 5.394e+05\n", " spatial_ref int32 32756\n", "Data variables:\n", " blue (time, y, x) int16 450 425 388 385 394 ... 408 382 367 364 383\n", " green (time, y, x) int16 488 454 414 406 408 ... 510 480 467 466 465\n", " red (time, y, x) int16 254 233 206 203 205 ... 329 292 272 277 273\n", " nir (time, y, x) int16 157 150 134 141 142 ... 160 144 137 150 165\n", " swir1 (time, y, x) int16 84 85 77 79 84 80 82 ... 111 82 80 72 81 97\n", " swir2 (time, y, x) int16 71 71 64 71 70 66 68 ... 97 73 65 60 65 80\n", " sdev (time, y, x) float32 0.007338 0.009259 ... 0.01109 0.009572\n", " edev (time, y, x) float32 267.4 232.7 172.5 ... 202.2 200.9 227.4\n", " bcdev (time, y, x) float32 0.2071 0.2088 0.1745 ... 0.1806 0.2081\n", " count (time, y, x) int16 13 13 12 12 12 12 12 ... 13 11 11 11 12 12\n", "Attributes:\n", " crs: PROJCS[\"WGS 84 / UTM zone 56S\",GEOGCS[\"WGS 84\",DATUM[\"WGS_...\n", " grid_mapping: spatial_ref" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ds_ls7 = dc.load(product=\"ga_ls7e_nbart_gm_cyear_3\",\n", " like=ds_ls8)\n", "\n", "ds_ls7" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Adding a progress bar\n", "When loading large amounts of data, it can be useful to view the progress of the data load. \n", "The `progress_cbk` parameter in `dc.load()` adds a progress bar that indicates how the load is progressing:\n", "\n", "![Progress bar](../Supplementary_data/04_Loading_data/progress_bar.jpg)\n", "\n", "The example cell below loads 5 years of data (2013, 2014, 2015, 2016 and 2017) from the `ls8_nbart_geomedian_annual` product with a progress bar:" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "3b377ce9e1644633a3ac066a2abaf588", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(HBox(children=(Label(value=''), Label(value='')), layout=Layout(justify_content='space-between'…" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "
\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "
<xarray.Dataset>\n",
       "Dimensions:      (time: 5, y: 424, x: 384)\n",
       "Coordinates:\n",
       "  * time         (time) datetime64[ns] 2013-07-02T11:59:59.999999 ... 2017-07...\n",
       "  * y            (y) float64 -3.156e+06 -3.156e+06 ... -3.168e+06 -3.168e+06\n",
       "  * x            (x) float64 2.067e+06 2.067e+06 ... 2.079e+06 2.079e+06\n",
       "    spatial_ref  int32 3577\n",
       "Data variables:\n",
       "    blue         (time, y, x) int16 490 506 504 497 498 ... 350 351 311 296 292\n",
       "    green        (time, y, x) int16 537 557 558 548 536 ... 526 531 450 392 391\n",
       "    red          (time, y, x) int16 253 273 275 269 268 ... 407 402 351 337 340\n",
       "    nir          (time, y, x) int16 93 106 104 100 108 ... 2889 2557 2344 2403\n",
       "    swir1        (time, y, x) int16 51 61 62 58 68 ... 1394 1397 1127 1009 1089\n",
       "    swir2        (time, y, x) int16 41 48 48 47 55 49 ... 661 650 521 465 495\n",
       "    sdev         (time, y, x) float32 0.003682 0.004547 ... 0.001669 0.002302\n",
       "    edev         (time, y, x) float32 142.1 160.1 159.9 ... 220.8 205.4 237.7\n",
       "    bcdev        (time, y, x) float32 0.1025 0.1248 0.1216 ... 0.04438 0.05313\n",
       "    count        (time, y, x) int16 10 11 11 11 12 11 11 ... 17 17 15 14 14 14\n",
       "Attributes:\n",
       "    crs:           EPSG:3577\n",
       "    grid_mapping:  spatial_ref
" ], "text/plain": [ "\n", "Dimensions: (time: 5, y: 424, x: 384)\n", "Coordinates:\n", " * time (time) datetime64[ns] 2013-07-02T11:59:59.999999 ... 2017-07...\n", " * y (y) float64 -3.156e+06 -3.156e+06 ... -3.168e+06 -3.168e+06\n", " * x (x) float64 2.067e+06 2.067e+06 ... 2.079e+06 2.079e+06\n", " spatial_ref int32 3577\n", "Data variables:\n", " blue (time, y, x) int16 490 506 504 497 498 ... 350 351 311 296 292\n", " green (time, y, x) int16 537 557 558 548 536 ... 526 531 450 392 391\n", " red (time, y, x) int16 253 273 275 269 268 ... 407 402 351 337 340\n", " nir (time, y, x) int16 93 106 104 100 108 ... 2889 2557 2344 2403\n", " swir1 (time, y, x) int16 51 61 62 58 68 ... 1394 1397 1127 1009 1089\n", " swir2 (time, y, x) int16 41 48 48 47 55 49 ... 661 650 521 465 495\n", " sdev (time, y, x) float32 0.003682 0.004547 ... 0.001669 0.002302\n", " edev (time, y, x) float32 142.1 160.1 159.9 ... 220.8 205.4 237.7\n", " bcdev (time, y, x) float32 0.1025 0.1248 0.1216 ... 0.04438 0.05313\n", " count (time, y, x) int16 10 11 11 11 12 11 11 ... 17 17 15 14 14 14\n", "Attributes:\n", " crs: EPSG:3577\n", " grid_mapping: spatial_ref" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "query = {\"x\": (153.3, 153.4),\n", " \"y\": (-27.5, -27.6),\n", " \"time\": (\"2013\", \"2017\")}\n", "\n", "ds_progress = dc.load(product=\"ga_ls8c_nbart_gm_cyear_3\",\n", " progress_cbk=with_ui_cbk(),\n", " **query)\n", "\n", "ds_progress" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Recommended next steps\n", "\n", "To continue working through the notebooks in this beginner's guide, the following notebooks are designed to be worked through in the following order:\n", "\n", "1. [Jupyter Notebooks](01_Jupyter_notebooks.ipynb)\n", "2. [Digital Earth Australia](02_DEA.ipynb)\n", "3. [Products and measurements](03_Products_and_measurements.ipynb)\n", "4. **Loading data (this notebook)**\n", "5. [Plotting](05_Plotting.ipynb)\n", "6. [Performing a basic analysis](06_Basic_analysis.ipynb)\n", "7. [Introduction to Numpy](07_Intro_to_numpy.ipynb)\n", "8. [Introduction to Xarray](08_Intro_to_xarray.ipynb)\n", "9. [Parallel processing with Dask](09_Parallel_processing_with_Dask.ipynb)\n", "\n", "Once you have worked through the beginner's guide, you can join advanced users by exploring:\n", "\n", "* A demonstration of how to load cloud-free observations in the [using load_ard](../How_to_guides/Using_load_ard.ipynb) notebook.\n", "* The \"DEA products\" directory in the repository, where you can explore DEA products in depth.\n", "* The \"How_to_guides\" directory, which contains a recipe book of common techniques and methods for analysing DEA data.\n", "* The \"Real_world_examples\" directory, which provides more complex workflows and analysis case studies." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "***\n", "## Additional information\n", "\n", "**License:** The code in this notebook is licensed under the [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0). \n", "Digital Earth Australia data is licensed under the [Creative Commons by Attribution 4.0](https://creativecommons.org/licenses/by/4.0/) license.\n", "\n", "**Contact:** If you need assistance, please post a question on the [Open Data Cube Slack channel](http://slack.opendatacube.org/) or on the [GIS Stack Exchange](https://gis.stackexchange.com/questions/ask?tags=open-data-cube) using the `open-data-cube` tag (you can view previously asked questions [here](https://gis.stackexchange.com/questions/tagged/open-data-cube)).\n", "If you would like to report an issue with this notebook, you can file one on [GitHub](https://github.com/GeoscienceAustralia/dea-notebooks).\n", "\n", "**Last modified:** December 2023\n", "\n", "**Compatible datacube version:** " ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1.8.6\n" ] } ], "source": [ "print(datacube.__version__)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Tags\n", "" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "**Tags**: :index:`sandbox compatible`, :index:`NCI compatible`, :index:`dc.load`, :index:`xarray.Dataset`, :index:`xarray.DataArray`, :index:`landsat 7`, :index:`landsat 8`, :index:`annual geomedian`, :index:`crs`, :index:`reprojecting data`, :index:`resampling data`, :index:`beginner`" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.10" }, "widgets": { "application/vnd.jupyter.widget-state+json": { "state": { "2bb6cd6b3ae24e62bad299675e0b8df3": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "LabelModel", "state": { "layout": "IPY_MODEL_913a803f0e6443429d8e6d629799d016", "style": "IPY_MODEL_c292a651c19a48079e8005d6539716fc", "value": "100 of 100" } }, "2c96bbdebc504f209c8396526a04292e": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "children": [ "IPY_MODEL_ba6b3d1f824d45c0aa095881ff23cbc9", "IPY_MODEL_2bb6cd6b3ae24e62bad299675e0b8df3" ], "layout": "IPY_MODEL_62c169993efd49568251d9a19bbbaa87" } }, "2f8bf8f637024b4bbf551f72fcce88ee": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "width": "100%" } }, "33bb73de6c9b41dd9ad11152d7df6e0e": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {} }, "3b377ce9e1644633a3ac066a2abaf588": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "VBoxModel", "state": { "children": [ "IPY_MODEL_2c96bbdebc504f209c8396526a04292e", "IPY_MODEL_f8d38ebd146c4724a9dc9d03f64a1220" ], "layout": "IPY_MODEL_d55147b344224e7da3aa68bd019a4f60" } }, "62c169993efd49568251d9a19bbbaa87": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "justify_content": "space-between" } }, "729bcfa54e6148dd9c349f67d67a95d2": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "IntProgressModel", "state": { "layout": "IPY_MODEL_2f8bf8f637024b4bbf551f72fcce88ee", "style": "IPY_MODEL_d8ae933b9e764c47b60c3a5e339b51f5", "value": 100 } }, "913a803f0e6443429d8e6d629799d016": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {} }, "9ee158f5c70145c1b3b505a5ba997baa": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "description_width": "" } }, "ba6b3d1f824d45c0aa095881ff23cbc9": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "LabelModel", "state": { "layout": "IPY_MODEL_33bb73de6c9b41dd9ad11152d7df6e0e", "style": "IPY_MODEL_9ee158f5c70145c1b3b505a5ba997baa", "value": "FPS: 3.5 (0.0 s remaining)" } }, "c292a651c19a48079e8005d6539716fc": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "description_width": "" } }, "cd25cdd871ff414580007b11018a99b4": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": {} }, "d55147b344224e7da3aa68bd019a4f60": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "width": "100%" } }, "d8ae933b9e764c47b60c3a5e339b51f5": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "description_width": "" } }, "f8d38ebd146c4724a9dc9d03f64a1220": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "children": [ "IPY_MODEL_729bcfa54e6148dd9c349f67d67a95d2" ], "layout": "IPY_MODEL_cd25cdd871ff414580007b11018a99b4" } } }, "version_major": 2, "version_minor": 0 } } }, "nbformat": 4, "nbformat_minor": 4 }