MESMA Documentation

Installation Instructions

For issues, bugs, proposals or remarks, visit the issue tracker.

Installation of QGIS Plugin

The plugin is available in the QGIS Python Plugins Repository:

Plugins menu > Manage and install plugins... > All > Search for 'MESMA'

Alternatively, you can download the latest stable distribution (mesma-x-qgis.zip) and install the plugin manually:

Plugins menu > Manage and install plugins... > Install from ZIP > Browse to the zip file > Click *Install plugin*.

Note

The plugin is build for QGIS Version 3.6 and up. We recommend to use QGIS version 3.10. The plugin has been tested on Windows 10.0, Ubuntu 16.04 and Raspbian GNU/Linux 10 (buster)

Installation of the python package

Open a shell by running the following batch script (adapt to match with your installation):

::QGIS installation folder
set OSGEO4W_ROOT=C:\OSGeo4W64

::set defaults, clean path, load OSGeo4W modules (incrementally)
call %OSGEO4W_ROOT%\bin\o4w_env.bat
call qt5_env.bat
call py3_env.bat

::lines taken from python-qgis.bat
set QGIS_PREFIX_PATH=%OSGEO4W_ROOT%\apps\qgis
set PATH=%QGIS_PREFIX_PATH%\bin;%PATH%

::make PyQGIS packages available to Python
set PYTHONPATH=%QGIS_PREFIX_PATH%\python;%PYTHONPATH%

:: GDAL Configuration (https://trac.osgeo.org/gdal/wiki/ConfigOptions)
:: Set VSI cache to be used as buffer, see #6448 and
set GDAL_FILENAME_IS_UTF8=YES
set VSI_CACHE=TRUE
set VSI_CACHE_SIZE=1000000
set QT_PLUGIN_PATH=%QGIS_PREFIX_PATH%\qtplugins;%OSGEO4W_ROOT%\apps\qt5\plugins

::enable/disable QGIS debug messages
set QGIS_DEBUG=1

::open the OSGeo4W Shell
@echo on
@if [%1]==[] (echo run o-help for a list of available commands & cmd.exe /k) else (cmd /c "%*")

For command-line interface and stand-alone usage, install the python package with pip:

pip install mesma

For offline installation, you can download the latest stable distribution (mesma-x.tar.gz) and

C:\WINDOWS\system32>cd C:\Users\UserName\Downloads
C:\Users\UserName\Downloads>pip install mesma-x.tar.gz

User Guide

MESMA can be used in several ways:

1. As a plugin in QGIS
2. As a command line interface in a terminal
3. Adapting the code to fulfil very specific needs

For the last option, we refer the user to the code repository and the API at the end of this document.

For issues, bugs, proposals or remarks, visit the issue tracker.

MESMA

Theoretical overview

A spectral mixing technique called Multiple Endmember Spectral Mixture Analysis (MESMA) is at the core of the plugin. MESMA is an extension of simple Spectral Mixture Analysis (SMA).

In simple SMA, a spectrum is modelled as the sum of pure spectra called endmembers, each weighted by the fraction of an endmember required to produce the mixture ([Adams1993], [Roberts1993], [Settle1993]). SMA has an advantage over many competing approaches in that it provides physically meaningful measures of cover that account for subpixel mixing.

While SMA is a powerful approach, it fails to account for pixel-scale variability in spectral dimensionality, spectral degeneracy between materials and natural variation in the spectra of most materials. The latter is often referred to as endmember variability (Somers et al. [Somers2011]). As a result, simple SMA under-utilizes the potential of most remote sensing data sets for discriminating materials, while at the same time producing fractional errors due to the incorrect type or number of endmembers used to unmix a specific pixel.

MESMA extends SMA by allowing the number and types of endmembers to vary on a per-pixel basis (Roberts et al. [Roberts1998]). MESMA overcomes limitations of SMA by requiring a model to meet minimum fit, fraction and residual constraints while testing multiple models for each image pixel.

MESMA is typically implemented by developing a spectral library, then unmixing an image using every possible combination of two, three, four or more endmembers applied to each pixel. Using this approach, significantly more than four materials can be mapped across an image, while minimizing pixel-scale fraction errors by selecting the best-fit model for each pixel.

Some of the earliest applications of MESMA were in snow-cover/snow-area mapping ([Painter1998]; [Painter2003]), plant species mapping in chaparral ([Roberts1998]; [Dennison2003a]; [Dennison2003b]; [Roberts2003]), urban remote sensing ([Rashed2003]; [Powell2007]; [Franke2009]), soil mapping in arid lands ([Okin2000]), improved estimation of LAI in boreal forests ([Sonnentag2007]), land form mapping in North Africa using MODIS ([Ballantine2005]), fire temperature mapping ([Dennison2006]; [Eckmann2009]), post-fire recovery in chaparral ([Peterson2003]), fire severity mapping ([Quintano2013]), surface coal-mine mapping ([Fernando2012]), shape-based temporal compositing ([Dennison2007]) and several planetary applications ([Li2003]; [Johnson2006]).

Since 2009, MESMA has become an increasingly valuable approach for classification and biophysical mapping. Somers et al. [Somers2009] introduced the concept of band weighting, in which endmember variability (Somers et al. [Somers2011]) is suppressed by selecting subsets of wavelengths that have the lowest impact on spectral fractions for a class (e.g., improved separability of endmembers). A key component of this work was the introduction of the Instability Index (ISI), which is calculated as the ratio of spectral variability within a class to spectral variability between classes, in which spectral variability is quantified using the coefficient of variation. The ideal wavelength subset will minimize the former and maximize the latter.

ISI was further extended to the concept of Stable Zone Unmixing (SZU; [Somers2010]), in which a band subset is selected that minimizes within class endmember variability and maximizes between class variability. Peterson et al. [Peterson2015] demonstrated that SZU improved the ability to discriminate oil-vegetation from NPV along the Gulf coast, following the Deepwater Horizon oil spill. Youngentob et al. [Youngentob2011] introduced the concept of using continuum-removed spectra and MESMA to discriminate Eucalyptus sub-genera. Variants of MESMA were applied to time series data by Somers and Asner ([Somers2013], [Somers2014]) to map invasive forest species in the Hawaiian islands.

Quantitative comparisons of reference fractions to estimates using SMA or MESMA have proven to be highly accurate over a diverse number of surfaces (e.g. [Elmore2000]; [Painter2003]; [Powell2007]; [Roberts2012]; [Roberts2015]; [Roberts2017]). Incorporating endmember variability, for example, was attributed by Roberts et al. [Roberts2012] as a primary factor leading to high fractional accuracy in mapping impervious surface fractions in urban areas.

Endmembers used in SMA or MESMA can be derived from an image (image endmembers), or from a library of known materials (reference endmembers). Reference endmembers can be derived from the field, laboratory, images or even radiative transfer models. Typical endmembers used in SMA include soil, green vegetation (GV), non-photosynthetic vegetation (NPV), impervious surfaces, and shade. Identifying a high quality set of reference or image endmembers is a critical stage of mixture modelling ([Tompkins1997]).

While MESMA is a powerful approach, several factors have limited its use. One of the major limitations has been a lack of easily used software. One of the key goals of the MESMA plugin is to address this lack of software. Another limitation has been the challenges in selecting endmembers to be used in MESMA. In its earliest implementations, MESMA was typically employed using hundreds of endmembers, resulting in significant challenges in interpretation and computation.

More recently, efforts have been made to fine tune the analysis by selecting optimal sets of endmembers from a spectral library prior to running MESMA (library pruning). A key goal is to identify those spectra that are most representative of a specific class, yet also least likely to be confused with spectra from a different class.

A number of approaches have been developed for identifying endmembers including the Pixel Purity Index (PPI) [Boardman1995], manual selection [Bateson1996], Endmember Bundles [Bateson2000], Constrained Reference Endmember Selection (CRES) [Roberts1993], Count-based Endmember Selection (CoB) [Roberts2003], Endmember Average RMSE (EAR) [Dennison2003a], Minimum Average Spectral Angle (MASA) [Dennison2004] and Iterative Endmember Selection (IES) [Roth2012].

Five techniques used for identifying optimal endmembers are included in the Spectral Library Tool and are discussed in detail on the tools website (https://spectral-libraries.readthedocs.io) : EAR, MASA and COB (collectively EMC), CRES and IES.

MESMA Tool

Once you have developed a spectral library, you can run mixture models using this tool. Again, the metadata belonging to the library is very important as it determines the classification of the spectra in the library.

The tool allows a user to select the model complexity, and to choose which classes to run and which models within a class to run. The tool also allows to set constraint options, including a minimum and maximum shade constraint.

The tool also performs multi-level fusion, allowing the user to adjust the RMSE threshold constraint.

The tool includes two shade options, photometric shade, in which the shade defaults to zero reflectance, or non-photometric shade, which enables a user to provide a shade spectrum from a spectral library.

Finally, two band selection options are included: spectral weighting as implemented in Somers et al. [Somers2009] and band selection using SZU [Somers2010].

In the case of multiple models, the best single model for each pixel will be selected based on minimum RMSE.

When running this tool, several items should be noted:

1. The number of bands and the band positions of the image and all spectral libraries used must be an exact match.
2. It is possible to run the same model on multiple images by selecting multiple, rather than a single, input file. When doing this, a user should turn off the option “Open images for Viewing” to minimize clutter and computational overhead. Output files will be assigned automatically in this case rather than being assigned by the user.

QGIS GUI

1. Select the spectral library.

   • The reflectance scale factor is automatically detected as 1, 1000 or 10000, but can be changed in the advanced settings.

     Note

     Reflectance data can be saved to file as reflectance values (i.e. values between 0 and 1) or the data can be multiplied by a scale factor (usually by 1000 or 10000). This is done to be able to store the data as integer values (no comma’s), because integers require less memory. Although this is more efficient storage-wise, it has as a result we have to divide the data values again before processing.

     The software tries to auto detect this scale factor, but does not always succeed (in the case of masked data with a lot of zero’s along the edges, or in the case where the scale factor is a different number, like 256). Double check this value in the Advances Settings tab.

   • Select the metadata element that divides the spectra in the library into classes.

   • The tool automatically selects all 2 and 3-EM models for you. To change the selected models, click Change…

     

     • Choose the model complexity (2-EM Models, 3-EM Models, …). For a classification exercise, a user would select a 2-EM Models (endmember + shade).

       Note

       The user can select as many complexity models as there are classes. However, be warned, the fundamental dimensionality of a pixel is usually no higher than four, and at times five, so selecting a nine endmember model would require considerable computation that most likely will reduce accuracy.

     • Within a complexity level, select which classes and class-models you would like to use.

     • The number at the bottom indicates the total number of models and warns the user if numbers get out of hand (green is good, red is bad). Mind though that this can vary with computer power and is mere an indication.

   • Go to the Advanced Settings if you want to change the Multi-level Fusion Threshold. This is the threshold that determines how much the RMSE must improve between model complexity levels, before the more complex model is selected. This defaults to 0.007 following multiple papers (e.g. [Roberts2012]).

   • The Advanced Settings also allow the user to include non-photometric shade.

2. Set the constraints (optional, but by default on):

   • Minimum endmember fraction: the lowest allowable fraction value. Cannot go below -0.5. Default -0.05.
   • Maximum endmember fraction: the highest allowable fraction value. Cannot go over 1.5. Default 1.05.
   • Minimum shade fraction: the lowest allowable shade fraction value. Setting the minimum will prevent negative shade fractions and can better discriminate endmembers by brightness. Default 0.
   • Maximum shade fraction: the highest allowable fraction value. Setting this can help to ensure that water and deep shadows remain un-modeled. Default 80%. However, recently Wetherley et al. [Wetherley2018] have demonstrated that a lower setting (20%) better discriminates between turf grass and trees.
   • RMSE constraint: the maximum allowable model RMSE.
   • In the Advanced Settings, it is also possible to set the Residual Constraint. It is composed of two components: a constraint threshold and a number of consecutive bands upon which to apply the constraint. Selecting a threshold of 0.025 and a count of 7, for example would exclude all models which produce residual spectra that exceed 0.025 for 7 consecutive bands. By default this option is not used, and in practice models with high RMSE also fail a residual constraint.

   Note

   Pixels that do not meet these constraints are classified as ‘unmodelled’ in the output.

3. Select the image or images that will be unmixed.

   • The reflectance scale factor is automatically detected as 1, 1000 or 10000, but can be changed in the advanced settings.

4. An output image name is automatically generated but can be changed when only one image is selected.

   • The default output name is InputImageName_mesma_YYYYMMDDThhmmss.
   • The Advanced Settings also allow the user to create a Residual Image. By default option is off as residuals images are large files and therefore require extra storage and incur computational costs to write them to file. Residual images contain the model residual spectra stored in floating point data type. For each pixel the residual spectrum is calculated as the image spectrum minus the model spectrum, where the model is the best fit model for that pixel as determined by minimum RMSE. Because the residuals image has the same number of bands as the input image and is a floating point image it is as large or larger than the input image.

5. Advanced MESMA algorithms are available in the last tab:

   • In Spectral Weighing (slow) [Somers2009], all bands are used, but a weighing factor is used to reduce the influence of bands that do not improve endmember separability.
   • Band Selection employs Stable Zone Unmixing (SZU) [Somers2010] and generates a reduced band set, where only those bands that best discriminate endmember pairs are retained.

Additional options:

• The Save Settings button allows the user to save all selected settings to a text file. This option is included to capture complicated settings which are time-consuming to input and prone to errors when reproducing.
• The Reload Settings button allows the user to reload the exact same model parameters at a later date to either modify them or to apply them to a different image.

Processing notes:

Note that if large images and/or a large number of models are used, this may require a long time to complete.

The use of a default naming convention based on time and date is very convenient. However, when running many models, it can result in a plethora of output files that differ only in the timing of the run. If you generate a product you like very much, and what to find it again easily, you might consider renaming

Issue Tracker:

For issues, bugs, proposals or remarks, visit the issue tracker.

CITATIONS

[Roberts2012]

Roberts DA, Quattrochi DA, Hulley GC, Hook SJ and Green RO. 2012. Synergies between VSWIR and TIR data for the urban environment: An evaluation of the potential for the Hyperspectral Infrared Imager (HyspIRI) Decadal Survey mission. Remote Sensing of Environment, volume 117, p. 83-101.

[Somers2009]

Somers B, Delalieux S, Stuckens J, Verstraeten WW and Coppin P. 2009. A weighted linear spectral mixture analysis approach to address endmember variability in agricultural production systems. International Journal of Remote Sensing, volume 30, p. 139-147.

[Somers2010]

Somers B, Delalieux S, Verstraeten WW, van Aardt JA, Albrigo L and Coppin P. 2010. An automated waveband selection technique for optimized hyperspectral mixture analysis. International Journal of Remote Sensing, volume 31, p. 5549-5568.

[Wetherley2018]

Wetherley EB, McFadden JP and Roberts DA. 2018. Megacity-scale analysis of urban vegetation temperatures. Remote Sensing of Environment, volume 213, p. 18-33.

Command Line Interface

The main command is:

>mesma

Use -h or --help to list all possible arguments:

>mesma -h

Only the spectral library file, the class name and the image are required arguments. An example:

>mesma data\spectral_library.sli LEVEL_2 data\image

All the other inputs are optional.

By default, the MESMA algorithm will run all 2- and 3-EM models. In case you would like to change this, use -l (the letter L, not number one) or --complexity-level. Keep in mind that 1-EM models do not exist and the maximum number of levels depends on the number of classes. An example:

>mesma data\spectral_library.sli LEVEL_2 data\image -l 2 3 4

The default fusion threshold is 0.007. Models with a higher number of classes (e.g. 4-EM vs. 3-EM models) are only chosen when the decrease in RMSE is larger than this threshold. In case you would like to change this, use -f or --fusion-threshold:

>mesma data\spectral_library.sli LEVEL_2 data\image -f 0.008

By default, the constraints are set at -0.05 (minimum endmember fraction), 1.05 (maximum endmember fraction), 0.00 (minimum shade fraction), 0.80 (maximum shade fraction) and 0.025 (maximum RMSE). Use --min-fraction, --max-fraction, --min-shade-fraction, --max-shade-fraction and --max-rmse to change these values.

Residual constraints are not used by default. To enable them, use --residual-constraint. This constraint specifies number of consecutive bands that the residual values are allowed to exceed a given threshold. The default values here are 0.025 (threshold) and 7 (number of consecutive bands). To change these values, use --residual-constraint-values and specify the two values in the order residual threshold and then number of bands.

To set no constraint for any given argument, use the value -9999. To use no constraints at all, use the argument -u or --unconstrained:

>mesma data\spectral_library.sli LEVEL_2 data\image --max-fraction 1.10 --max-rmse -9999
>mesma data\spectral_library.sli LEVEL_2 data\image --residual-constraint --residual-constraint-values 0.02 10
>mesma data\spectral_library.sli LEVEL_2 data\image -u

Use -a or --shade to set non-photometric shade. This is a spectral library with a single shade spectrum:

>mesma data\spectral_library.sli LEVEL_2 data\image -a data\shade_library

The reflectance scale of the image, spectral library and shade spectrum can all be automatically derived from the values in the array as 1, 1 000 or 10 000. In case you would like to specify another scale, use -r or --reflectance-scale-library to set the reflectance scale factor of the spectral library; -s or --reflectance-scale-image to set the reflectance scale factor of the image; and -t or --reflectance-scale-shade to set the reflectance scale factor of the shade spectrum:

>mesma data\spectral_library.sli LEVEL_2 data\image --r 255 --s 255

By default, the output file is stored in the same folder as the image file, with the extension ‘_mesma_’ followed by a timestamp. To select another file or another location, use the argument -o or --output:

>mesma data\spectral_library.sli LEVEL_2 data\image -o "data\subfolder\mesma_image"

It is also possible to create a residual image as part of the output. By default this is disabled. To turn the option on, use -d or --residuals-image:

>mesma data\spectral_library.sli LEVEL_2 data\image -d

Lastly, there are 2 advance MESMA algorithms part of the tool: Spectral Weighing and Band Selection (=Stable Zone Unmixing). To turn on Spectral Weighing, use --spectral-weighing. To turn on Stable Zone Unmixing, use --band-selection. The Stable Zone Unmixing algorithm also required two input variables: the correlation threshold (default 0.99) and the correlation decrease (default 0.01). To change these values, use --band-selection-values to specify the tuple (correlation threshold, correlation decrease):

>mesma data\spectral_library.sli LEVEL_2 data\image --band-selection --band-selection-values 0.80 0.05

Issue Tracker:

For issues, bugs, proposals or remarks, visit the issue tracker.

MESMA Output

The output file names are all built, by default, like: ImageName_mesma_YYYMMDDThhmmss. There are 3 images and 2 metadata files:

• a models image (models given in the form of the endmember number of the spectral library)
• a fraction image (_fractions)
• an RMSE image (_rmse)

The models image is not human readable and should be opened with the MESMA Visualisation Tool.

MESMA Visualisation Tool

As a user, we want to interpret the results of MESMA. Where can we find what class? Where can we find what mixture? We could analyse the “models” image band by band (i.e. class by class), but this is rather tedious.

A better option is to use the Visualisation Tool included in the toolbox:

• From the drop down menu, select the models image.
• In case the layer is not yet open in QGIS, use the Add Raster button to open it.
• Make sure it is the top layer in the table of contents menu, or the only layer turned on.
• Automatically, all classes and class-models are displayed in a tree-structure.
• Now you can turn classes on and off and change their colours.
• View the result on the map canvas by using the Show button .
• In case you are not satisfied with the colours, you can undo all changes with the Refresh button .

As a user, you must be aware that a pixel can contain multiple classes. It is therefore not very useful to try and display several classes (e.g. “green vegetation” or “soil”) at the same time. Because of the overlap, only the top class will be shown on the map canvas.

ACKNOWLEDGMENTS

This user guide is based on the VIPER Tools 2.0 user guide (UC Santa Barbara, VIPER Lab): Roberts, D. A., Halligan, K., Dennison, P., Dudley, K., Somers, B., Crabbe, A., 2018, Viper Tools User Manual, Version 2, 91 pp.

CITATIONS

[Adams1993]

Adams JB, Smith MO and Gillespie, AR. 1993. Remote geochemical analysis: elemental and mineralogical composition. Chapter: Imaging spectrometry: interpretation based on spectral mixture analysis. Cambridge University Press, volume 7, p. 145-166.

[Ballantine2005]

Ballantine JAC, Okin GS, Prentiss DE and Roberts DA. 2005. Mapping African landforms using continental scale unmixing of MODIS imagery. Remote Sensing of Environment, volume 97, p. 470-483.

[Bateson1996]

Bateson A and Curtiss B. 1996. A method for manual endmember selection and spectral unmixing. Remote Sensing of Environment, volume 55, p. 229-243.

[Bateson2000]

Bateson CA, Asner GP and Wessman C. 2000. Endmember Bundles: A new approach to incorporating endmember variability into spectral mixture analysis. IEEE Transactions on Geoscience and Remote Sensing, volume 38, p. 1083-1094.

[Boardman1995]

Boardman JW, Kruse FA and Green RO. 1995. Mapping target signatures via partial unmixing of AVIRIS data. Summaries of the 5th JPL Airborne Earth Science Workshop, JPL Publication 95, p. 23-26.

[Dennison2003a]

(1, 2) Dennison PE, Roberts DA. 2003. Endmember Selection for Multiple Endmember Spectral Mixture Analysis using Endmember Average RSME. Remote Sensing of Environment, volume 87, p. 123-135.

[Dennison2003b]

Dennison PE, Roberts DA. 2003. The Effects of Vegetation Phenology on Endmember Selection and Species Mapping in Southern California Chaparral. Remote Sensing of Environment, volume 87, p. 295-309.

[Dennison2004]

Dennison PE, Halligan KQ and Roberts DA. 2004. A Comparison of Error Metrics and Constraints for Multiple Endmember Spectral Mixture Analysis and Spectral Angle Mapper. Remote Sensing of Environment, volume 93, p. 359-367.

[Dennison2006]

Dennison PE, Charoensiri K, Roberts DA, Peterson SH and Green RO. 2006. Wildfire temperature and Land Cover Modeling Using Hyperspectral Data. Remote Sensing of Environment, volume 100, p. 212-222.

[Dennison2007]

Dennison PE, Roberts DA, Peterson SH. 2007. Spectral shape-based temporal compositing algorithms for MODIS surface reflectance data. Remote Sensing of Environment, volume 109, p. 510-522.

[Eckmann2009]

Eckmann T, Roberts DA and Still C. 2009. Estimating subpixel fire sizes and temperatures from ASTER using multiple endmember spectral mixture analysis. International Journal of Remote Sensing, volume 30, p. 5851-5864.

[Elmore2000]

Elmore AJ, Mustard JF, Manning SJ and Lobell DB. 2000. Quantifying vegetation change in semiarid environments: precision and accuracy of spectral mixture analysis and the normalized difference vegetation index}. Remote Sensing of Environment, volume 73, p. 87-102.

[Fernando2012]

Fernando-Manso A, Quintano C and Roberts DA. 2012. Evaluation of potential of multiple endmember spectral mixture analysis (MESMA) for surface coal mining affected area mapping in different world forest ecosystems. Remote Sensing of Environment, volume 127, p. 181-192.

[Franke2009]

Franke J, Roberts DA, Halligan K and Menz G. 2009. Hierarchical Multiple Endmember Spectral Mixture Analysis (MESMA) of hyperspectral imagery for urban environments. Remote Sensing of Environment, volume 113, p. 1712-1723.

[Johnson2006]

Johnson JR, Staid MI, Titus TN, Becker K. 2006. Shocked plagioclase signatures in Thermal Emission Spectrometer Data of Mars. Icarus, volume 180, p. 60-74.

[Li2003]

Li L and Mustard JF. 2003. Highland contamination in lunar mare soils: Improved mapping with multiple end-member spectral mixture analysis. Journal of Geophysical Research, volume 108, p. 7-1 to 7-14.

[Okin2000]

Okin GS, Roberts DA, Murray B and Okin WJ. 2000. Practical limits on hyperspectral vegetation discrimination in arid and semiarid environments. Remote Sensing of Environment, volume 77, p. 212-225.

[Painter1998]

Painter TH, Roberts DA, Green RO and Dozier J. 1998. The Effect of Grain Size on Spectral Mixture Analysis of Snow-covered Area from AVIRIS Data. Remote Sensing of Environment, volume 65, p. 320-332.

[Painter2003]

(1, 2) Painter TH, Dozier J, Roberts DA, Davis RE and Green RO. 2003. Retrieval of Subpixel Snow-covered Area and Grain Size from Imaging Spectrometer Data. Remote Sensing of Environment, volume 85, p. 64-77.

[Peterson2003]

Peterson SH and Stow DA. 2003. Using multiple image endmember spectral mixture analysis to study chaparral regrowth in southern California. International Journal of Remote Sensing, volume 24, p. 4481-4504.

[Peterson2015]

Peterson SH, Roberts DA, Beland M, Kokaly RF and Ustin SL. 2015. Oil detection in the coastal marshes of Louisiana using MESMA applied to band subsets of AVIRIS data. Remote Sensing of Environment, volume 159, p. 222-231.

[Powell2007]

(1, 2) Powell R, Roberts DA, Dennison PE and Hess LL. 2007. Sub-pixel mapping of urban land cover using multiple endmember spectral mixture analysis: Manaus, Brazil. Remote Sensing of Environment, volume 106, p. 253-267.

[Quintano2013]

Quintano C, Fernando-Manso A and Roberts DA. 2013. Multiple Endmember Spectral Mixture Analysis (MESMA) to map burn severity levels from Landsat images in Mediterranean countries. Remote Sensing of Environment, volume 136, p. 76-88.

[Rashed2003]

Rashed T, Weeks JR, Roberts DA, Rogan J and Powell R. 2003. Measuring the Physical Composition of Urban Morphology using Multiple Endmember Spectral Mixture Models. Photogrammetric Engineering and Remote Sensing, volume 69, p. 1011-1020.

[Roberts1993]

(1, 2) Roberts DA, Adams JB and Smith MO. 1993. Discriminating Green Vegetation, Non-Photosynthetic Vegetation and Soils in AVIRIS Data. Remote Sensing of Environment, volume 44, p. 255-270.

[Roberts1998]

(1, 2) Roberts DA, Gardner M, Church R, Ustin S, Scheer G and Green RO. 1998. Mapping Chaparral in the Santa Monica Mountains using Multiple Endmember Spectral Mixture Models, Remote Sensing of Environment, volume 65, p. 267-279.

[Roberts2003]

(1, 2) Roberts DA, Dennison PE, Gardner M, Hetzel Y, Ustin SL and Lee C. 2003. Evaluation of the Potential of Hyperion for Fire Danger Assessment by Comparison to the Airborne Visible/Infrared Imaging Spectrometer. IEEE Transactions on Geoscience and Remote Sensing, volume 41, p. 1297-1310.

[Roberts2012]

(1, 2) Roberts DA, Quattrochi DA, Hulley GC, Hook SJ and Green RO. 2012. Synergies between VSWIR and TIR data for the urban environment: An evaluation of the potential for the Hyperspectral Infrared Imager (HyspIRI) Decadal Survey mission. Remote Sensing of Environment, volume 117, p. 83-101.

[Roberts2015]

Roberts DA, Dennison PE, Roth KL, Dudley K and Hulley G. 2015. Relationships Between Dominant Plant Species, Fractional Cover and Land Surface Temperature in a Mediterranean Ecosystem. Remote Sensing of Environment, volume 167, p. 152-167.

[Roberts2017]

Roberts DA, Alonzo M, Wetherley E, Dudley K and Dennison P. 2017. Scale in Remote Sensing and GIScience Applications. Chapter: Multiscale Analysis of Urban Areas using Mixing Models, p. 247-282.

[Roth2012]

Roth KL, Dennison PE and Roberts DA. 2012. Comparing endmember selection techniques for accurate mapping of plant species and land cover using imaging spectrometer data. Remote Sensing of Environment, volume 127, p. 139-152.

[Settle1993]

Settle JJ and Drake NA. 1993. Linear mixing and the estimation of ground cover proportions. International Journal of Remote Sensing, volume 14, p. 1159-1177.

[Somers2009]

(1, 2) Somers B, Delalieux S, Stuckens J, Verstraeten WW and Coppin P. 2009. A weighted linear spectral mixture analysis approach to address endmember variability in agricultural production systems. International Journal of Remote Sensing, volume 30, p. 139-147.

[Somers2010]

(1, 2) Somers B, Delalieux S, Verstraeten WW, van Aardt JA, Albrigo L and Coppin P. 2010. An automated waveband selection technique for optimized hyperspectral mixture analysis. International Journal of Remote Sensing, volume 31, p. 5549-5568.

[Somers2011]

(1, 2) Somers B, Asner GP, Tits L and Coppin P. 2011. Endmember Variability in Spectral Mixture Analysis: A review. Remote Sensing of Environment, volume 115, p. 1603-1616.

[Somers2013]

Somers B and Asner GP. 2013. Multi-temporal hyperspectral mixture analysis and feature selection for invasive species mapping in rainforests. Remote Sensing of Environment, volume 136, p. 14-27.

[Somers2014]

Somers B and Asner GP. 2014. Tree species mapping in tropical forests using multi-temporal imaging spectroscopy: wavelength adaptive spectral mixture analysis. International Journal of Applied Earth Observation and Geoinformation, volume 31, p. 57-66.

[Sonnentag2007]

Sonnentag O, Chen JM, Roberts DA, Talbot J, Halligan KQ and Govind A. 2007. Mapping tree and shrub leaf area indices in an ombrotrophic peatland through multiple endmember spectral unmixing. Remote Sensing of Environment, volume 109, p. 342-360.

[Tompkins1997]

Tompkins S, Mustard JF, Pieters CM and Forsyth DW. 1997. Optimization of endmembers for spectral mixture analysis. Remote Sensing of Environment, volume 59, p. 472-489.

[Youngentob2011]

Youngentob KN, Roberts DA, Held AA, Dennison PE, Jia X and Lindenmayer DB. 2011. Mapping two Eucalyptus subgenera using multiple endmember spectral mixture analysis and continuum-removed imaging spectrometry data. Remote Sensing of Environment, volume 115, p. 1115-1128.

Post-processing

Shade Normalisation of SMA product

Shade normalization can be performed on SMA fraction images to assess relative abundance of surface materials with the shade component removed. For example, assume we are interested in the relative abundance of green vegetation in a semi-arid region with high topographic relief. In the standard SMA outputs the contribution of topographic shade may cause some confusion when interpreting the green vegetation abundance values. If comparing the green vegetation fractions between a region with low local incidence angle and a region with high local incidence angle, the region of low local incidence will show a higher green vegetation fraction due to the reduced shade component, even if the green vegetation abundance is the same as the other site. Shade normalization is particularly valuable in urban areas, where dark impervious surface is often mapped as having a high shade fraction. These effects can be removed by shade normalizing the SMA output images.

QGIS GUI

1. Select the input SMA fraction image.
2. Optionally, the user can specify an output name.

The output image will contain one less band the input image since the shade band will no longer be present, but the endmember fractions for each pixel will still sum to 1.

Issue Tracker:

For issues, bugs, proposals or remarks, visit the issue tracker.

Command Line Interface

The main command is:

>mesma-shade

Use -h or --help to list all possible arguments:

>mesma-shade -h

Only the fractions image is a required argument. It is the MESMA output image ending on _fractions. An example:

>mesma-shade data\output\mesma_fractions

By default, the output file is stored in the same folder as the image file, with the extension ‘_normalised’. To select another file or another location, use the argument -o or --output:

>mesma-shade data\output\mesma_fractions -o data\other\normalised_image

Issue Tracker:

For issues, bugs, proposals or remarks, visit the issue tracker.

Soft to Hard Classification of SMA product

The SMA fraction image gives a very detailed insight in the sub-pixel fractions in the images landscape. This tool is more suited for user interested in a hard classification, assigning the class of the most abundant fraction (e.g. GV or SOIL, but excluding shade) to any given pixel.

The output image will contain a single band with numbers ranging from 0 to the number of classes minus one. If you open the image in QGIS, the description (in the layer properties) will reveal what class each number stands for.

QGIS GUI

1. Select the input SMA fraction image.
2. Optionally, the user can specify an output name.

The output image will contain a single band with numbers ranging from 0 to the number of classes minus one. If you open the image in QGIS, the description (in the layer properties) will reveal what class each number stands for.

Issue Tracker:

For issues, bugs, proposals or remarks, visit the issue tracker.

Command Line Interface

The main command is:

>mesma-classify

Use -h or --help to list all possible arguments:

>mesma-classify -h

Only the fractions image is a required argument. It is the MESMA output image ending on _fractions. An example:

>mesma-classify data\output\mesma_fractions

By default, the output file is stored in the same folder as the image file, with the extension ‘_classification’. To select another file or another location, use the argument -o or --output:

>mesma-classify data\output\mesma_fractions -o data\other\classification_image

Issue Tracker:

For issues, bugs, proposals or remarks, visit the issue tracker.

ACKNOWLEDGMENTS

This user guide is based on the VIPER Tools 2.0 user guide (UC Santa Barbara, VIPER Lab): Roberts, D. A., Halligan, K., Dennison, P., Dudley, K., Somers, B., Crabbe, A., 2018, Viper Tools User Manual, Version 2, 91 pp.

Exercises

For issues, bugs, proposals or remarks, visit the issue tracker.

We have developed two exercises. One with a data set in Brussels (Belgium), and one with a data set in Santa Barbara, CA (USA).

These exercises consist of two parts: creating and optimizing spectral libraries (visit http://spectral-libraries.readthedocs.io) and MESMA and post-processing: this part you will find here.

Exercise: Brussels

For issues, bugs, proposals or remarks, visit the issue tracker.

Objectives

1. Learn to work with the MESMA toolbox in QGIS
2. Use MESMA as a sub-pixel classification method:

• Interpret the results visually
• Compare performance when using different libraries
• Compare different MESMA techniques
• Perform a hard classification on your MESMA result

Tutorial Data Set

You can download the tutorial data set here (tutorial_data_set_brussels.zip). The zip file contains the following data:

• Apex images from 2015 with numbers 014, 14 and 180 in ENVI format
• A spectral library in ENVI format
• A validation shape file for each image
• Note: images and library have been smoothed using Savitzky-Golay filter with window size 9

Acknowledgement for the data set:

Degerickx, Roberts, Somers; 2019; Enhancing the performance of Multiple Endmember Spectral Mixture Analysis (MESMA) for urban land cover mapping using airborne lidar data and band selection; Volume 221; P 260-273

Note

It is good practice to keep all files in the same folder - especially during the exercises. Files like square arrays often go looking for library information on which they are built.

Image Inspection

1. Try to visualize the images in QGIS.

To recognize the surroundings, overlay them with a map from the OpenStreetMap project (QuickMapServices plugin and Google Satellite View).

2. Inspect the technical properties of the image.

• Why are the images black when first loading them into QGIS? Which bands would you use to visualize them for easy interpretation? Look-up the wavelengths of the RGB bands.
• What is the size of the image and of each pixel?
• Make a list of the land cover classes you expect to find in each image.

Creating and Optimization of Spectral Libraries

See exercises on http://spectral-libraries.readthedocs.io.

MESMA

In SMA, a mixed pixel (image) is modelled as a linear combination of endmembers (library spectra) and their fractions.

MESMA wants to account for within-class spectral variability and allows a single land cover class to be represented by multiple endmembers. As a result, the selected EMs vary on a per-pixel basis, allowing dynamic mapping of spectrally complex scenes.

Exercise: MESMA

For issues, bugs, proposals or remarks, visit the issue tracker.

We will unmix our images with two different libraries and compare the results:

1. The library we created with the IES pruning technique.
2. The library we created by manually selecting spectra from the image.

Once we have selected the best library, we can do unmixing on several levels:

1. We start with a pixel classification (based on 2-EM models).
2. Then we follow with 3-EM and 4-EM models.

Data Used

Use the image apex_2015_180_smooth and the libraries library_ies.sli and library_from_image.sli (you might have given these libraries a different name!).

We will always use the standard constraints of 0.025 RMSE, -0.05 and 1.05 minimum and maximum fraction constraints, and 0 to 0.80 shade constraints.

Part 1: Basic MESMA with the IES pruned library

Spectral Library:

• Select the library library_ies.sli.
• Select the metadata class named Meta2.
• The algorithm automatically selects all 2- and 3-EM models. We leave this as it is.
• Check on the Advanced Settings tab that the Reflectance scale factor is automatically set to 1.

Constraints:

• We don’t change any constraints.

Input image:

• Select the image apex_2015_180_smooth.

Run MESMA:

• We leave the output name as is and make sure the output opens in QGIS.
• Click RUN.

Expect it to take a few moments. Don’t be alarmed if QGIS freezes for the duration of the calculations.

Results

The output RMSE can be found in the image named apex_2015_180_smooth_mesma_…_rmse.

Initially this image is very dark. Set the min and max values to 0 and 0.025 in the layer properties to get a more dynamic image. The highest error values seem to be on some roofs.

Open the validation shape file and compare your results. The folder also includes a style file for the validation shape file.

The fractions are in the image named apex_2015_180_smooth_mesma_…_fractions. This is a multi-band image, but only is useful if each band is viewed separately.

The models image is the most interesting from a visual analysis point of view. The models image comes with a csv file and both are not directly useful for analysis. We need the MESMA Visualisation Tool to analyse them. Normally it opens automatically. Otherwise click .

• The Visualization Tool shows two menus: Classes and Class-models. The Classes show you the pixels that have a given class in their model, whether it is part of a mixture or found solely. The Class-Models show the different models you chose when you were setting up MESMA.
• The number of pixels indicate how often a model was used.
• You can turn on/off classes and class-models, but each time you want to see the changes on the image, you have to use the View button .
• You can change the colors of your classes and class-models, but you have to select a row, right click and then choose ‘Set Color’. This is a bit confusing as you might also by accident change the color of other previously selected rows.

We already see a quite good result. There is some confusion between pavement and roofs, but vegetation is classified rather well. Mind though that this is a very basic visualisation, drawing one class on top of another! Select each class individually to see where they are found! Then compare this with the fraction image to see if the presence of each class is significant or not.

Part 2: Basic MESMA with the manually created library

Now we perform the same exercise, but with the library we created by selecting spectra manually from the image.

Results

A lot of unmodeled pixels in this image. The library was oversimplified.

What is your conclusion? Was the unmixing successful? Explain why there is so much confusion between pavement and roofs, and between shrubs and trees.

Go beyond

We can try a bunch of other things:

• Use the full (unpruned) library: this will be much slower, but will it be better?
• Use the advanced MESMA techniques like Stable Zone Unmixing or Band Weighing
• Include 4-EM or 5-EM Models: this will slow down your search, but will the results improve?

Post-Processing: Soft to Hard Classification

Lastly, we can do a soft to hard classification based on the Fraction Image (under post-processing tools): each pixel will be assigned the dominant fraction (excluding shade).

Exercise: Santa Barbara

For issues, bugs, proposals or remarks, visit the issue tracker.

Tutorial Data Set

You can download the tutorial data set here (tutorial_data_set_santa_barbara.zip). The zip file contains the following data:

• Image 010614r4_4-5_clip

  A 2001 AVIRIS image (ENVI header format) with 20 m spatial resolution ([Roberts2003]).

• Image demo1.sub

  A 2011 AVIRIS image (ENVI header format) as 600x600 pixel subset of downtown Santa Barbara ([Roberts2015], [Roberts2017]. AVIRIS data were processed to apparent surface reflectance using ATCOR, as described in [Roberts2015], and scaled to reflectance times 10,000.

• Spectral Library: roberts2017_urban.sli

  This library was used in [Roberts2017] and consists of 3725 spectra. The spectra include a mixture of spectra extracted from the three AVIRIS flight lines acquired in 2011 merged with the urban spectral library described by [Herold2004]. The AVIRIS spectra, which originally consisted of over 60,000 spectra, were subsampled using code developed by Keely Roth as part of her dissertation. Following the protocols described in [Roth2012], spectra were sampled at the polygon level such that no polygon can contribute more than 10 spectra to the library, or more than 50% of the pixels for small polygons. The p9 in the file name, implies this was the 9th random pull out of 10 pulls. The 9th pull was selected because it produced the highest accuracy (based on IES) with the lowest number of spectra, as described in [Roberts2017].

• Spectral Library: 010627_westusa_all.sli

  Spectral Library for the CRES tutorial and for Simple SMA.

• Spectral Library: roberts_et_al_2017_final.sli.

  This is the same library that was used to generate the final fraction maps in Roberts et al. (2017). The library was created by merging 2,739 spectra extracted from the image with 986 spectra measured in the field. IES was run, generating a 497 spectral subset. This library was further pruned to remove poorly performing models to generate a 376 endmember subset, then all clearly mixed spectra (assessed visually), were removed to generate the nomix library, consisting of 259 spectra. Finally, a procedure, designed to remove spectrally degenerate spectra was used to generate a 91 endmember subset. The final culling was accomplished in 8 iterations.

• roi.shp

  This contains regions of interest for most dominant cover types in the region. This matches demo1.sub.

Acknowledgement for the data set:

Roberts, D. A., Halligan, K., Dennison, P., Dudley, K., Somers, B., Crabbe, A., 2018, Viper Tools User Manual, Version 2, 91 pp.

Note

It is good practice to keep all files in the same folder - especially during the exercises. Files like square arrays often go looking for library information on which they are built.

Creating and Optimization of Spectral Libraries

See exercises on http://spectral-libraries.readthedocs.io.

MESMA

Exercise: MESMA in QGIS

For issues, bugs, proposals or remarks, visit the issue tracker.

While many algorithms can classify images, MESMA’s strength is in modeling mixtures of endmembers across multiple levels of complexity, ranging from simple mixtures of a single endmember and shade (two endmember model) to more complex mixtures requiring a total of three or four endmembers. When used for more complex mixing models, the fractions for each endmember within selected models become a valuable output from MESMA.

However, the trade off of more complex models is a much larger number of potential models being compared, which lengthens run times.

The procedure we often use is to start with a two-endmember model to classify the image, than prune that model to produce a better performing subset. This subset can be then used as a library for a three, four or five endmember model.

However, keep in mind several important facts:

• A library that works well for two-endmember classification may include some endmembers that are mixtures.
• Some classes which are suitable for a two-endmember classification may not be suitable at higher levels of complexity. Two good examples in this tutorial include water, which will be mapped in many lower reflectance areas, and artificial turf, which is a pretty rare class and probably does not form many three or more endmember mixtures.
• Large libraries can generate a very large number of models. For example, a library with 10 models in each of 6 classes will generate 60 2-EM, 1500 3-EM and 20000 4-EM models! Reducing the number of endmembers and/or the number of classes will reduce run times.
• Many spectra which may represent a viable class at one level may be spectrally degenerate at a higher level of complexity and thus have little actual impact on the fractions. A model becomes spectrally degenerate when it can be unmodeled as a combination of two other endmembers or is spectrally similar enough to another endmember it has little impact on the fractions.

We will start with a classification, using MESMA to classify the images based on two endmember models. We will evaluate model performance using the metadata files and remove any poorly performing models. Then we will follow with a three and four endmember model where we will only examine fraction performance.

Data used

We will unmix two images:

• 010614r4_4-5_clip: an AVIRIS scene from 1998, used for simple SMA with no fraction constraints.
• demo1.sub: a subset from the 2011 image from downtown Santa Barbara.

We will use the libraries that were created in the CRES, EMC and IES exercises:

• 010627_westusa_cres.sli
• roberts2017_urban_emc_opt.sli
• roberts2017_urban_ies.sli
• roberts_et_al_2017_final.sli

Simple SMA with no fraction constraints

To run Simple SMA, you can run as many endmembers as is reasonable given the dimensionality of the data, although four is typical.

Most often users will model GV, NPV, Soil and Shade for vegetated landscapes. For urban landscapes, an impervious surface endmember is often used. You are free to experiment with other endmember combinations (say muddy water vs clean water or impervious vs soil, or ash vs soil).

In most simple SMA cases, we prefer to run models with only one constraint on the fractions, namely that they sum to 1. Negative or super positive fractions are allowed.

Spectral Library:

• Select the library 010627_westusa_cres.sli.
• Select the metadata class named Class.
• The algorithm automatically selects all 2- and 3-EM models. We change this to use only the 4-EM complexity level. This results in 1 model.
• Check on the Advanced Settings tab that the Reflectance scale factor is automatically set to 1000.

Constraints:

• We don’t set any constraints.

Input image:

• Select the image 010614r4_4-5_clip

Run MESMA:

• We can change the output name to mesma_exercise1 and make sure the output opens in QGIS.
• Click OK.

Expect it to take a few minutes. Don’t be alarmed if QGIS freezes for the duration of the calculations.

The output RMSE can be found in the image named _rmse and the fractions are in the image named _fractions.

You will note that the fraction image looks good, but has a fairly large number of super positive and negative fractions. The NPV and GV fractions are most reasonable in senescent grasslands and chamise, but tend towards low to negative NPV in riparian areas, and oak forests (Clearly a limitation of CCEOLSTCK). Golf courses are particularly bad.

The RMSE is generally low (often less than 1%), but highest in urban areas (often over 2.5%). Given that our best model selections were for chamise, grassland and soils, these results are not unexpected. To better model oaks, we would need a better reference endmember than the ones we had in the reference library.

Running MESMA as a classifier with a two-endmember model

MESMA can be used as a classification algorithm, by picking the two endmember model (one endmember + shade) that fits each pixel spectrum with the lowest RMSE. Run in this way, MESMA can account for variations in the brightness of pixel spectra, but not mixing between multiple material types unless endmembers representing those mixtures are included in the endmember library.

In addition to the fractions of the non-shade and shade endmembers, MESMA will output the class of the best fit non-shade endmember to provide a classified image.

Spectral Library:

• Select the library roberts2017_urban_emc_opt.sli (this is the library we created with EMC).
• Select the metadata class named Level_2. This will give us eight classes: gv, npv, soil, rock, roof, paved and water.
• The algorithm automatically selects all 2- and 3-EM models. We change this to use only the 2-EM complexity level. This results in around 150 models.

Constraints:

We will accept physically reasonable fractions (min=0, max=1.0), physically reasonable shade (min=0,max=0.80) and set the RMSE constraint to 0.025.

Input image:

• Select the image demo1.sub.

Run MESMA:

• We can change the output name to mesma_exercise2 and make sure the output opens in QGIS.
• Click OK.

Expect it to take a few minutes. Don’t be alarmed if QGIS freezes for the duration of the calculations.

Remove under-performing models

A two-endmember model, generated using a model pruned either using EMC, IES or some combination, is not necessarily the ideal model. First and foremost, the utility of the model depends on how representative the library is of the image. If the library is very good, minimal pruning may be necessary.

However, if the pruning technique has either selected models that are extremely rare in the image, or models that tend to be confused with materials outside of their class, the classes will either be under-mapped or over-mapped. Either case is a problem, but we also have the tools on hand to address both.

The first approach is to cull out under-performing models. For this, we will use the MESMA Models Viewer.

A small portion of the image (a little under 50k pixels) remained unclassified. By sorting the models on number of pixels, we can easily find several under-performing models.

As an exercise, you can find all poorly performing models (for this exercise let’s say 20 or less pixels). You’ll find around 30 models. Simultaneously open the spectral library to analyze the models.

Some poorly performing models include:

• GV: MIRRA STAR_JASMINE
• PAVED: TILE_ROAD ASPHALT_ROAD CONCRETE_ROAD
• ROOF: ASPHALT_ROOF WOOD_SHINGLE_ROOF TILE_ROOF
• NPV: WOOD NPV (LITTER)
• SOIL: ASPHALT_GRAVEL

Some caution should be used in discarding a model, in case it is appropriate for another image.

Remove models that are modeling incorrect classes

The previous exercise works well for discarding models that do not model much area. However, it does not tell us anything about models that are modeling a large area of the wrong material.

This has to be done by also examining the RMSE and fraction images:

• load one image for the RMSE
• load one image for each fraction band
• load the original image as RGB or false color composite (1650 nm, 830 nm, and 650 nm as R, G, and B)

The image below shows from left to right, examples of the RMSE, GV, NPV and Paved fractions (using single band gray).

Examining the image, we see that qualitatively, the fractions look good. Areas of obvious GV are mapped as GV, the same goes for NPV and soil. Roofs, paved areas and rock as a group are mapped correctly, but seem to be mixed a lot.

Try to identify the most problematic rock spectrum. First visualize all rock models and visualize them one by one. RDP_117_X2486_Y202 seems to be map a lot of highway.

You can do this for all data types. Obviously, this kind if pruning is only possible with an in-depth knowledge of the study area.

Following is our own assessment of bad models: - BAPI05_X344_Y234 - CISP_09_X632_Y378: tends to model many urban trees and shrubs - EUSP_I04_X290_Y243: over maps Eucalyptus - MARSH01_X405_Y302 - BRNI_I02_X695_Y802: possibly okay, certainly not safe to discard - concrete_bridge_1_X2667_Y700: tough call, mapping paved surfaces, but not concrete bridges - tennis_court_n1_X1321_Y399: mapping roads

Using the IES library

Do the exact same exercise with the IES library at Level 2 and display the results. You can also display the fraction images as RGB images, using three different fractions for R, G and B, e.g. NPV-GV-Soil and Paved-Roof-Rock, like in the images below:

We see another model that looks, qualitatively, to be quite good. Areas of green vegetation are modeled with high GV fraction and senescent grassland is mapped in the north. Paved looks quite good with roads mapped in red. Roof looks generally good, but at least one area is probably soil mapped as roof.

We will use a combination of the MESMA Model Viewer and the Spectral Library to evaluate model performance, as we dit before. Right away we find quite some under-performing models (around 60), especially with roof an paved spectra.

To find over-mapping models, carefully compare the false color composite to the NPV-GV-Soil image to find areas that clearly look to be mapped wrong.

Generating the final libraries

Having identified poorly performing models with the EMC and IES libraries, you can now remove them using the spectral libraries tool.

Running MESMA for a three endmember model

Spectral Library:

• Select the library roberts_et_al_2017_final.sli, which has much of its spectral degeneracy removed
• Select the metadata class named Level_2.
• The algorithm automatically selects all 2- and 3-EM models.

Constraints:

We will accept physically reasonable fractions (min=0, max=1.0), physically reasonable shade (min=0,max=0.80) and set the RMSE constraint to 0.025.

Input image:

• Select the image demo1.sub.

Run MESMA:

• We can change the output name to mesma_exercise4 and make sure the output opens in QGIS.
• Click OK.

Expect it to take a few minutes. Don’t be alarmed if QGIS freezes for the duration of the calculations.

A high proportion of the image was modeled, with only 1695 pixels un-modeled. Much of the un-modeled area is water. Examine the models by displaying a false color composite, NPV-GV-Soil fractions, Paved-Roof-Rock fractions and the RMSE.

The images below show the RMSE and the NPV-GV-Soil fractions images:

Overall, this as a very satisfactory model. Looking at the RMS error image, we see a few areas of high RMS, one of which some miss-mapping between paved an roof. This suggests to that we are short a critical endmember. Water is also an area of high error, which is normal, as water was taken out of the model.

Running MESMA for a four endmember model

Spectral Library:

• Select the library roberts_et_al_2017_final.sli
• Select the metadata class named Level_2.
• The algorithm automatically selects all 2- and 3-EM models. Also select 4-EM models. MESMA warns us that we will run 53 637 models. Depending on your RAM and processors, this will likely take more than two hours.

Constraints:

We will accept physically reasonable fractions (min=0, max=1.0), physically reasonable shade (min=0,max=0.80) and set the RMSE constraint to 0.025.

Input image:

• Select the image demo1.sub.

Run MESMA:

• We can change the output name to mesma_exercise5 and make sure the output opens in QGIS.
• Click OK.

Load a false color composite, NPV-GV-Soil Paved-Roof-Soil and RMS images. Qualitatively this image looks quite good. Unlike the EMC version, soils appear to be well mapped and the RMSE is substantially lower throughout. This is a good model.

Looking at the metadata, we see that this model captured all but 7,900 pixels, leaving only 2.2% of the image unmodeled (and this is mostly water). There are no substantial fraction errors that are evident.

Grasslands are often modeled with two endmembers, orchards with three endmembers, and the urban areas often require four endmember models.

Exercise: Stable Zone Unmixing

Stable Zone Unmixing will compare pairs of classes to determine a subset of wavelengths that maximizes spectral separability between the classes, while minimizing within class variability (Somers et al., 2010). In principle, this should produce increased fraction accuracy, while decreasing run times because fewer wavelengths are being used.

Spectral Weighting (the other option), should, in principle, generate a similar result, but is likely to be significantly slower because it retains all wavelengths in the mixing process, while adding the calculations of weighting factors, so it is likely to be even slower than the original three endmember model.

Data used

• demo2.sub: a subset from the 2011 image from Goleta, Santa Barbara.
• roberts_et_al_2017_final.sli

Exercise

We will run this unmixing twice, once with and once without Stable Zone Unmixing, in order to be able to compare the result:

Spectral Library:

• Select the library roberts_et_al_2017_final.sli
• Select the metadata class named Level_2.
• The algorithm automatically selects all 2- and 3-EM models.

Constraints:

We will accept physically reasonable fractions (min=0, max=1.0), physically reasonable shade (min=0,max=0.80) and set the RMSE constraint to 0.025.

Input image:

• Select the image demo2.sub.

Advanced MESMA Algorithms:

• The second time toggle on Use band selection. You can leave the default correlation values as they are.

Run MESMA:

• We can change the output name to demo2_szu and make sure the output opens in QGIS.
• Click OK.

Examine your model output as we did before, loading a false color composite, NPV-GV-Soil fraction image, Paved-Roof-Rock fraction image and RMSE.

Comparing this to the original model run, we see they are qualitatively very similar. In a few areas in the southeast, the RMS appears slightly higher in this version. However, we also see that soils mapping has improved, with soil fractions now modeled in areas we know to be soil, in some areas previously mapped as roof. SZU tends to favor higher complexity models, whereas using all wavelengths tends to favor simpler models. Improved performance for mapping soils is much more evident by direct comparison.

The image below shows the original model on the left and the SZU model on the right for NPV-GV-Soil and Paved-Roof-Rock. An arrow marks one area where the models differ significantly.

The SZU version modeled all but 7k pixels, taking under 2 minutes. The original model had more un-modeled pixels (10k) and took over 8 minutes. At this stage, we do not know which model is more accurate, although we would suggest that the SZU version may be slightly more accurate. However, the SZU version is significantly faster. Assuming the factor of seven in run time improvement is consistent, our four-endmember model would have taken 31 minutes instead of 278.

We can also verify that SZU selects for higher complexity by comparison to the original model using all wavelengths. The figure on the left shows that the original model modeled most of the image with two endmember models (blue), whereas the SZU version tended to favor three endmember models.

For issues, bugs, proposals or remarks, visit the issue tracker.

CITATIONS

[Herold2004]

Herold M, Roberts DA, Gardner ME and Dennison PE. 2004. Spectrometry for urban area remote sensing – Development and analysis of a spectral library from 350 to 2400 nm. Remote Sensing of Environment, volume 91, p. 304-319.

[Roberts2003]

Roberts DA, Dennison PE, Gardner M, Hetzel Y, Ustin SL and Lee C. 2003. Evaluation of the Potential of Hyperion for Fire Danger Assessment by Comparison to the Airborne Visible/Infrared Imaging Spectrometer. IEEE Transactions on Geoscience and Remote Sensing, volume 41, p. 1297-1310.

[Roberts2015]

(1, 2) Roberts DA, Dennison PE, Roth KL, Dudley K and Hulley G. 2015. Relationships Between Dominant Plant Species, Fractional Cover and Land Surface Temperature in a Mediterranean Ecosystem. Remote Sensing of Environment, volume 167, p. 152-167.

[Roberts2017]

(1, 2, 3) Roberts DA, Alonzo M, Wetherley E, Dudley K and Dennison P. 2017. Scale in Remote Sensing and GIScience Applications. Chapter: Multiscale Analysis of Urban Areas using Mixing Models, p. 247-282.

[Roth2012]

Roth KL, Dennison PE and Roberts DA. 2012. Comparing endmember selection techniques for accurate mapping of plant species and land cover using imaging spectrometer data. Remote Sensing of Environment, volume 127, p. 139-152.

MESMA API

Source code: https://bitbucket.org/kul-reseco/mesma/src.

For issues, bugs, proposals or remarks, visit the issue tracker.

MESMA

MESMA Module

———————————————————————————————————————-

Date : August 2018

Copyright : © 2018 - 2020 by Ann Crabbé (KU Leuven)

Email : acrabbe.foss@gmail.com

Acknowledgements : Translated from VIPER Tools 2.0 (UC Santa Barbara, VIPER Lab).

Dar Roberts, Kerry Halligan, Philip Dennison, Kenneth Dudley, Ben Somers, Ann Crabbé

This file is part of the MESMA plugin and python package.

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public

License as published by the Free Software Foundation, either version 3 of the License, or any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied

warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License (COPYING.txt). If not see www.gnu.org/licenses.

———————————————————————————————————————-

class mesma.core.mesma.MesmaCore(n_cores: int = 1)

Bases: object

Multiple Endmember Signal Mixture Analysis: calculate SMA for multiple endmember combinations and select the best fit based on the lowest RMSE.

Citations: MESMA: Roberts, D.A., Gardner, M., Church, R., Ustin, S., Scheer, G., Green, R.O., 1998, Mapping Chaparral in the Santa Monica Mountains using Multiple Endmember Spectral Mixture Models, Remote Sensing of Environment, 65, p. 267-279.

Multilevel fusion: Roberts, D.A., Dennison, P.E., Gardner, M., Hetzel, Y., Ustin, S.L., Lee, C., 2003, Evaluation of the Potential of Hyperion for Fire Danger Assessment by Comparison to the Airborne Visible/Infrared Imaging Spectrometer, IEEE Transactions on Geoscience and Remote Sensing, 41, p. 1297-1310.

Spectral band weighing: Somers, B., Delalieux, S, Stuckens, J , Verstraeten, W.W, Coppin, P., 2009, A weighted linear spectral mixture analysis approach to address endmember variability in agricultural production systems, International Journal of Remote Sensing, 30, p. 139-147.

Spectral band selection: Somers, B., Delalieux, S., Verstraeten, W.W., van Aardt, J.A.N., Albrigo, G., Coppin, P., 2010, An automated waveband selection technique for optimized hyperspectral mixture analysis. International Journal of Remote Sensing, 31, p. 5549-5568.

execute(image: numpy.array, library: numpy.array, look_up_table: dict, em_per_class: dict, constraints: list = (-0.05, 1.05, 0.0, 0.8, 0.025, -9999, -9999), fusion_value: float = 0.007, no_data_pixels: tuple = (), shade_spectrum: numpy.array = None, residual_image: bool = False, use_band_weighing: bool = False, use_band_selection: bool = False, bands_selection_values: tuple = (0.99, 0.01), log: callable = <built-in function print>) → tuple

Execute MESMA. Process input and output.

In case band weighing or band selection algorithms are used, no residual image or residual constraints can be used.

Returns 3 images [a pixels * b pixels * c bands]:

• the best model [nb of bands = nb of classes] - each band contains the library spectra number per class
• the model’s fractions [nb of bands = nb of classes + 1], including a shade fraction
• the model’s RMSE [nb of bands = 1]
• [optional] a residual image

Value of unmodeled pixels in output:

• models: -1
• fractions: 0
• rmse: 9999
• residual_image: 0

Value of pixels with no data in output:

• models: -2
• fractions: 0
• rmse: 9998
• residual_image: 0

Parameters:

• image – image, scaled to reflectance, without bad bands
• library – spectral library with spectra as columns, scaled to reflectance, without bad bands
• look_up_table – all endmember combinations (=models) for MESMA; ordered per complexity level and per class-model; n_models x n_endmembers
• em_per_class – a list of all library indices per endmember class
• constraints – min + max endmember fraction, min + max shade fraction, max rmse, residual reflectance threshold + max number of consecutive bands exceeding threshold. set value to -9999 if not used.
• no_data_pixels – indices of pixels that contain no data (result of np.where)
• shade_spectrum – single spectrum of photometric shade
• fusion_value – only select a model of higher complexity (e.g. 3-EM over 2-EM) of the RMSE is better with at least this value
• residual_image – output the residuals as an image (ignored when using band weighing or -selection)
• use_band_weighing – use the weighted linear spectral mixture analysis (Somers et al, 2009)
• use_band_selection – use the bands selection algorithm (Somers et al, 2010)
• bands_selection_values – correlation threshold and decrease for the band selection algorithm
• log – log function

Returns:

images with the best model for each pixel, the model fractions and rmse belonging {+ evt. residuals)

class mesma.core.mesma.MesmaModels

Bases: object

Create the MESMA look-up-table from a list of classes and user input. No GUI/CLI.

DEFINITIONS:

• endmember: spectrum or signal from a Spectral Library = ‘EM’
• class: logical group of endmembers, e.g. ‘green vegetation’ or ‘soil’
• endmember-model: combination of endmembers used for unmixing
• class-model: endmember-models grouped by class-level, e.g. all ‘green vegetation-soil’ models
• level: class-models grouped by the number of classes (e.g. all 3-EM models)

max_digits() → int

Get the maximum number of digits for the GUI, in order to be able to display the number of models.

Returns:The maximum number of digits for the GUI, in order to be able to display the number of models.

return_look_up_table() → dict

Get the actual look-up-table as a dictionary with a key-value pair for each level. Each level’s value is another dictionary with a key-value pair for each class-model. Each class-model has a numpy array of all the endmember-combinations for that class-model.

This look-up-table is a required input for MESMA.

Returns:The look-up-table as a dictionary [levels] of dictionaries [class-models] of numpy-arrays [models].

save() → str

Get a summary of the selected models, designed specifically for the ‘SAVE’ functionality in the MESMA gui.

Returns:A summary of the selected models, specifically to save in the MESMA settings [multi-line-str].

select_class(state: bool, index: int, level: int)

Add/remove a class of a given level. Automatically selects all class-models of all selected classes. For level 3 (= 3-EM models), at least 2 classes must be selected, etc.

Parameters:

• state – True = select, False = deselect.
• index – The index of the class, based on the list of unique lowercase classes.
• level – The complexity level in which the user wants to select.

select_level(state: bool, level: int)

Add/remove a level from the selection.

• Selecting a level for the first time does not automatically select any classes/class-models in that level.
• Deselecting a level leaves all settings of class/class-model selections intact.
• Selecting a level for the second time (or more) re-instated the previous settings of that level.

Parameters:

• state – True = select, False = deselect.
• level – The complexity level the user wants to select.

select_model(state: bool, index: int, level: int)

Add/remove an individual class-model.

Parameters:

• state – True = select, False = deselect.
• index – The index of the class-model
• level – The complexity level in which the user wants to select.

setup(class_list: numpy.array)

Default set up of the model selection: select all 2-EM and 3-EM models.

Parameters:class_list – [array-of-strings] A class for each endmember in the library.

summary() → str

Get a summary of the selected models. This is designed specifically to display in the GUI application.

Returns:A summary of the selected models, specifically to display in a GUI application [multi-line-str].

total() → int

Get the total number of models in the current selection.

Returns:The total number of models in the current selection.

total_per_class_model(model: tuple) → int

Get the total number of models of a given class-model in the current selection.

Parameters:model – The class-model. Returns:The total number of models of a given class-model in the current selection.

total_per_level(level: int) → int

Get the total number of models of a given level in the current selection.

Parameters:level – The complexity level. Returns:The total number of models of a given level in the current selection.

Post-processing

Shade Normalisation

———————————————————————————————————————-

Date : September 2018

Copyright : © 2018 - 2020 by Ann Crabbé (KU Leuven)

Email : acrabbe.foss@gmail.com

Acknowledgements : Translated from VIPER Tools 2.0 (UC Santa Barbara, VIPER Lab).

Dar Roberts, Kerry Halligan, Philip Dennison, Kenneth Dudley, Ben Somers, Ann Crabbé

This file is part of the MESMA plugin and python package.

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public

License as published by the Free Software Foundation, either version 3 of the License, or any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied

warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License (COPYING.txt). If not see www.gnu.org/licenses.

———————————————————————————————————————-

class mesma.core.shade_normalisation.ShadeNormalisation

Bases: object

Do a simple shade normalization on an existing MESMA fraction image: the shade fraction is removed and the other fractions are normalized to sum to one.

static execute(mesma_fraction_image: numpy.array, shade_band: int = -1) → numpy.array

Execute shade normalisation. The return value is a normalized fraction image, with one band less as the input.

Parameters:

• mesma_fraction_image – Fraction image of the output of the MESMA algorithm.
• shade_band – Band number with the shade fraction (set to None in case no shade band given).

Returns:

Normalised fraction image.

Soft To Hard Classification

———————————————————————————————————————-

Date : February 2019

Copyright : © 2019 - 2020 by Ann Crabbé (KU Leuven)

Email : acrabbe.foss@gmail.com

This file is part of the MESMA plugin and python package.

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public

License as published by the Free Software Foundation, either version 3 of the License, or any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied

warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License (COPYING.txt). If not see www.gnu.org/licenses.

———————————————————————————————————————-

class mesma.core.hard_classification.HardClassification

Bases: object

Do a simple hard classification on an existing MESMA fraction image.

static execute(mesma_fraction_image: numpy.array, shade_band: int = -1) → numpy.array

Execute the hard classification. The return value single band image indicating the dominant class.

Parameters:

• mesma_fraction_image – Fraction image of the output of the MESMA algorithm.
• shade_band – Band number with the shade fraction (set to None in case no shade band given).

Returns:

Hard classified image with values reflecting the dominant band.

See the User Guide for instructions on using the Command Line Interface.

About MESMA

MESMA is both a QGIS plugin and stand-alone python package that implements the MESMA (Multiple Endmember Spectral Mixture Analysis) unmixing algorithm in the field of Remote Sensing.

It is based on VIPER Tools: a software package written for ENVI/IDL and released in 2007. Several updates have been released since and now it has been ported to PyQGIS in the period 2017 - 2020. The original VIPER Tools is now split over two python/QGIS tools: Spectral Library Tool and MESMA.

Spectral Library Tool provides the following:

• Creating spectral libraries interactively (selecting spectra from an image or using regions of interest) and managing the metadata (developed by HU Berlin)
• Optimizing spectral libraries with Iterative Endmember Selection, Ear-Masa-Cob or CRES
• Website: <https://spectral-libraries.readthedocs.io>
• Repository: https://bitbucket.org/kul-reseco/spectral-libraries

MESMA provides the following:

• Running SMA and MESMA (with multi-level fusion, stable zone unmixing, …)
• Post-processing of the MESMA results (visualisation tool, shade normalisation, …)
• Website: <https://mesma.readthedocs.io>
• Repository: https://bitbucket.org/kul-reseco/mesma

The software has now been developed in the open source environment to encourage further development of the tool.

PLEASE GIVE US CREDIT

When using the MESMA tool, please use the following citation:

Crabbé, A. H., Somers, B., Roberts, D. A., Halligan, K., Dennison, P., Dudley, K. (2020). MESMA QGIS Plugin (Version x) [Software]. Available from https://bitbucket.org/kul-reseco/mesma.

ACKNOWLEDGEMENTS

The software and user guide are based on VIPER Tools 2.0 (UC Santa Barbara, VIPER Lab): Roberts, D. A., Halligan, K., Dennison, P., Dudley, K., Somers, B., Crabbe, A., 2018, Viper Tools User Manual, Version 2, 91 pp.

MESMA is also using two python packages developed by Benjamin Jakimow and Andreas Rabe:

• QGISPluginSupport (Benjamin Jakimow, HU Berlin): https://bitbucket.org/jakimowb/qgispluginsupport
• hub-datacube (Andreas Rabe, HU Berlin): https://bitbucket.org/hu-geomatics/hub-datacube

This revision is funded primarily through BELSPO (the Belgian Science Policy Office) in the framework of the STEREO III Programme – Project LUMOS - SR/01/321.

Logo’s were created for free at https://logomakr.com.

SOFTWARE LICENSE

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License (COPYING.txt). If not see www.gnu.org/licenses.

Indexes

• Index
• Module Index