OSD Signal Decompositions#

Signal Decompositions Module for OSD

This module contains standardized signal decomposition models for use in the SDT algorithms using the OSD modeling language. The defined signal decompositions are:

  1. _osd_l2_l1d1_l2d2p365: separating a piecewise constant component from a smooth and seasonal component, with Gaussian noise

    • l2: gaussian noise, sum-of-squares small or l2-norm squared

    • l1d1: piecewise constant heuristic, l1-norm of first order differences

    • l2d2p365: small second order diffs (smooth) and 365-periodic

  2. _osd_tl1_l2d2p365: similar to (2), estimating a smooth, seasonal component with an asymmetric laplacian noise model, fitting a local quantile instead of a local average

    • tl1: ‘tilted l1-norm,’ also known as quantile cost function

    • l2d2p365: small second order diffs (smooth) and 365-periodic

  3. _osd_l1_l1d1_l2d2p365: like (1) but with an asymmetric residual cost instead of Gaussian residuals

    • l1: l1-norm

    • l1d1: piecewise constant heuristic, l1-norm of first order differences

    • l2d2p365: small second order diffs (smooth) and 365-periodic

  4. _osd_l2_l1d2_constrained:

    • l2: gaussian noise, sum-of-squares small or l2-norm squared

    • l1d2: piecewise linear heuristic

    • constrained to have first val at 0 and last val at 1

solardatatools._osd_signal_decompositions._osd_l1_l1d1_l2d2p365(signal, use_ixs=None, w1=1.0, w2=0.001, transition_locs=None, return_all=False, solver=None, sum_card=False, verbose=False)#

Used in solardatatools/algorithms/capacity_change.py

This is a nonconvex problem when invoking QSS and sum_card=True.

Parameters:

  • signal – A 1d numpy array (must support boolean indexing) containing the signal of interest

  • use_ixs – List of booleans indicating indices to use in signal. None is default (uses the entire signal).

  • w1 – The regularization parameter to control the number of breakpoints in the PWC component

  • return_all – Returns all components and the objective value. Used for tests.

  • solver – Solver to use for the decomposition. QSS and OSQP are supported with OSD. MOSEK will trigger CVXPY use.

  • sum_card – Boolean for using the nonconvex formulation using the cardinality penalty, Supported only using OSD with the QSS solver.

  • verbose – Sets verbosity

Returns:

A tuple with three 1d numpy arrays containing the three signal component estimates

solardatatools._osd_signal_decompositions._osd_l2_l1d1_l2d2p365(signal, w1=1, w2=0.001, return_all=False, transition_locs=None, yearly_periodic=False, solver='QSS', use_ixs=None, sum_card=False, verbose=False)#

Used in: solardatatools/algorithms/time_shifts.py

This performs total variation filtering with the addition of a seasonal baseline fit. This introduces a new signal to the model that is smooth and periodic on a yearly time frame. This does a better job of describing real, multi-year solar PV power data sets, and therefore does an improved job of estimating the discretely changing signal. Default solver is QSS, and timeshift algorithm takes the final solution from solving the nonconvex problem with sum_card=True.

Parameters:

  • signal – A 1d numpy array (must support boolean indexing) containing the signal of interest

  • w1 – The regularization parameter to control the total variation in the final output signal

  • w2 – The regularization parameter to control the smoothness of the seasonal signal

  • yearly_periodic – Adds periodicity constraint to signal decomposition

  • return_all – Returns all components and the objective value. Used for tests.

  • solver – Solver to use for the decomposition

  • sum_card – Boolean for using the nonconvex formulation using the cardinality penalty, Supported only using OSD with the QSS solver.

  • verbose – Sets verbosity

Returns:

A tuple with two 1d numpy arrays containing the two signal component estimates

solardatatools._osd_signal_decompositions._osd_l2_l1d2_constrained(signal, w0=1, w1=5, return_all=False, solver='OSQP', verbose=False)#

Used in solardatatools/algorithms/clipping.py

This is a convex problem and the default solver across SDT is OSQP.

Parameters:

  • signal – A 1d numpy array (must support boolean indexing) containing the signal of interest

  • w0 – Weight on the residual component

  • w1 – The regularization parameter on l1d2 component

  • return_all – Returns all components and the objective value. Used for tests.

  • solver – Solver to use for the decomposition

  • verbose – Sets verbosity

Returns:

A tuple with returning the signal, the l1d2 component estimate, and the weight

solardatatools._osd_signal_decompositions._osd_tl1_l2d2p365(signal, use_ixs=None, tau=0.75, w0=1, return_all=False, solver='OSQP', verbose=False)#
Used in:

solardatatools/algorithms/sunrise_sunset_estimation.py solardatatools/clear_day_detection.py solardatatools/data_quality.py solardatatools/sunrise_sunset.py

This is a convex problem and the default solver across SDT is OSQP.

Parameters:

  • signal – A 1d numpy array (must support boolean indexing) containing the signal of interest

  • use_ixs – List of booleans indicating indices to use in signal. None is default (uses the entire signal).

  • tau – Quantile regression parameter,between zero and one, and it sets the approximate quantile of the residual distribution that the model is fit to See: https://colab.research.google.com/github/cvxgrp/cvx_short_course/blob/master/applications/quantile_regression.ipynb

  • w0 – Weight on the residual component

  • w1 – The regularization parameter to control the smoothness of the seasonal signal

  • yearly_periodic – Adds periodicity constraint to signal decomposition

  • return_all – Returns all components and the objective value. Used for tests.

  • solver – Solver to use for the decomposition

  • verbose – Sets verbosity

Returns:

A tuple with three 1d numpy arrays containing the three signal component estimates