Physical Quantities and Uncertanties

Any physical measurement is not a simple number. It is a number, but have a physical unit and an uncertainty related to it.

astropy has a very useful module called units, that handles physical units and Quantity, a container intended to store physical measurements with units.. But it doesn’t perform any error propagation.

To handle this and make the things a lot easier in astropop, we created a new class, called QFloat to handle both uncertainties and physical units at once. By QFloat we mean “Quantity Float”, relating it to physical measurements. This class mainly wraps units methods to handle units and perform error propagation in a coherent way. This module is used in a lot of places inside astropop, mainly in processing and FrameData to ensure correct processing in terms of units and errors propagation.

Warning

In the actual state, QFloat assumes all uncertainties are standard deviation errors, performing the standard error propagation method. The error propagation also assumes that the uncertainties are uncorelated. This is fine for our usage, but may present problems if used out of this context.

The QFloat Class

The QFloat class stores basically 3 variables: the nominal value, the uncertainty and the physical unit. To create this class, just pass these values in the class constructor and you can access it using the proper properties.

In [1]: from astropop.math import QFloat

In [2]: import numpy as np

# A physical measure of 1.000+/-0.001 meters
In [3]: qf = QFloat(1.0, 0.001, 'm')

In [4]: qf.nominal # the nominal value, must be 1.0
Out[4]: 1.0

In [5]: qf.uncertainty # the uncertainty, 0.001
Out[5]: 0.001

In [6]: qf.unit # astropy's physical unit, meter
Out[6]: Unit("m")

In [7]: qf # full representation.
Out[7]: 
<QFloat at 0x7f754aab8d90>
1.000+-0.001 m

Note that, for the full representation of the quantity, the nominal and the uncertainty values are rounded to the first non-zero error decimal. Internally, the number is stored with all the computed decimal places and this rounding just appears in the string representation.

In [8]: qf = QFloat(1.0583225, 0.0031495756, 'cm')

In [9]: qf.nominal
Out[9]: 1.0583225

In [10]: qf.uncertainty
Out[10]: 0.0031495756

In [11]: qf
Out[11]: 
<QFloat at 0x7f754b40be50>
1.058+-0.003 cm

QFloat also can store arrays of data, using the exactly same behavior.

In [12]: qf = QFloat([1.0, 2.0, 3.0], [0.1, 0.2, 0.3], 'm/s')

In [13]: qf
Out[13]: 
<QFloat at 0x7f754a8dc910>
[1.0+-0.1, 2.0+-0.2, 3.0+-0.3] unit=m / s

During the creation, you can omit uncertainty or unit arguments, but not the nominal value. We decided to don’t make possible create empty QFloat instances. Omiting arguments, the code interprets it as:

• unit: setting None unit, or omiting it, the class automatically interpret it as dimensionless_unscaled. This means: a number without physical unit.

In [14]: qf_nounit = QFloat(1.0, 0.1)

In [15]: qf_nounit
Out[15]: 
<QFloat at 0x7f754ad97410>
1.0+-0.1

• uncertainty: setting uncertainties as None, or omiting it, the code consider automatically the uncertainty as 0.0.

In [16]: qf_nostd = QFloat(1.0, unit='m')

In [17]: qf_nostd
Out[17]: 
<QFloat at 0x7f754aa91650>
1.0+-0.0 m

You also can omit both, like converting a single dimensionless number to QFloat.

In [18]: qf = QFloat(1.0)

In [19]: qf
Out[19]: 
<QFloat at 0x7f754a9ffe90>
1.0+-0.0

Printing and String Representation

By default, the QFloat shows the numbers rounded to the first significant digit of the uncertainty (rounded). This is done to make the representation more readable. But you can set the number of significant digits to show by setting the sig_digits property.

In [20]: qf = QFloat(1.0583225, 0.0031495756, 'cm')

In [21]: print(qf)
1.058+-0.003 cm

In [22]: qf.sig_digits = 2

In [23]: print(qf)
1.0583+-0.0031 cm

In [24]: qf.sig_digits = 3

In [25]: print(qf)
1.05832+-0.00315 cm

Units and Conversion

Physical units are fully handled with units module. But we don’t use Quantity class to store the files. Instead, the code perform itself the units checking and conversions. This is needed to make it compatible with uncertainties, but reduces the compatibility with astropy functions directly.

Internal conversion of units for math operations are made automatically and don’t need manual intervention. But manual conversions are also possible using the << operator or to function. The same do the same thing: convert the actual QFloat to a new one, with the ne unit. Both accept UnitBase instance or string for conversion.

In [26]: from astropop.math import QFloat

In [27]: from astropy import units

# One kilometer with 1 cm uncertainty
In [28]: qf = QFloat(1000, 0.01, 'm')

In [29]: qf << 'km'
Out[29]: 
<QFloat at 0x7f754b1292d0>
1.00000+-0.00001 km

In [30]: qf << units.cm
Out[30]: 
<QFloat at 0x7f754a266cd0>
100000+-1 cm

If improper conversion (incompatible units), an UnitConversionError is raised.

Math Operations and Error Propagation

As the main porpouse of this module, mathematical operations using physical quantities are performed between QFloat and compatible classes. We ensure a basic set of math operations to work, specially all the basic and trigonometric operations needed for basic data reduction. The code also support some basic numpy array functions, but not all of them.

The operations are performed with proper unit management and conversion (when necessary), and simplyfied uncorrelated error propagation. For a function \(f\) of \(x, y, z, \cdots\) variables, the error \(\sigma\) associated to each one of them, is propagated using the common equation:

\[\sigma_f = \sqrt{ \left(\frac{\partial f}{\partial x}\right)^2 \sigma_x^2 + \left(\frac{\partial f}{\partial y} \right)^2 \sigma_y^2 + \left(\frac{\partial f}{\partial z} \right)^2 \sigma_z^2 + \cdots}\]

Note that, for this simplyfied version of the error propagation equation, all variables are assumed to be independent and errors uncorrelated. All the error propagation is done by uncertainties, that supports some error correlations. However, due to the way we have to handle the operations wrapping with units, it’s expected that these correlated errors don’t work well in our code.

Supported Math Operations

Since the math operations are the main reason of the QFloat to exist, they have a special focus in the implementation. All builtin Python math operations, with the exception of matrix multiplication, is implemented for QFloat. This makes possible to perform direct math operations with QFloat.

As example, to sum two QFloat, you just need to use the + operator.

In [31]: qf1 = QFloat(1.0, 0.1, 'm')

In [32]: qf2 = QFloat(2.0, 0.1, 'm')

In [33]: qf1 + qf2
Out[33]: 
<QFloat at 0x7f754aab3250>
3.0+-0.1 m

astropop handles all the needed units checking and conversions and the result is dimensionality correct. For example:

In [34]: qf1 = QFloat(60, 0.5, 'km')

In [35]: qf2 = QFloat(7000, 700, 'm')

In [36]: qf1 + qf2
Out[36]: 
<QFloat at 0x7f754a8871d0>
67.0+-0.9 km

In [37]: t = QFloat(2.0, 0.1, 'h')

In [38]: qf1/t
Out[38]: 
<QFloat at 0x7f754a2df090>
30+-2 km / h

Incorrect dimensionality in operations will raise UnitsError.

In [39]: qf1 = QFloat(3.0, 0.01, 'kg')

In [40]: qf2 = QFloat(5.0, 0.2, 'K')

In [41]: qf1 + qf2
UnitConversionError: Can only apply 'decorator' function to quantities with compatible dimensions

Supported Numpy Array Operations

Some Numpy array functions are supported built-in by the QFloat, but not all of them. These functions are intended to perform array manipulations, like append, reshape, transpose, etc. With this compatibility you can use the Numpy functions directly with QFloat objects, like:

In [42]: qf = QFloat(np.zeros((100, 100)), np.zeros((100, 100)))

In [43]: np.shape(qf)
Out[43]: (100, 100)

One big difference from our compatibility to default Numpy is that for some functions, Numpy return the view of the array, for bigger performance. Our method, however, just return copies of the QFloat with applied functions. The impact im performance is not so big and the memory usage will not be a problem, unless you use a very very large array.

The current Numpy array functions supported for this class are:

• append

• around

• clip

• delete

• expand_dims

• flip

• fliplr

• flipud

• insert

• moveaxis

• ravel

• repeat

• reshape

• resize

• roll

• rollaxis

• round

• rot90

• shape

• size

• squeeze

• swapaxes

• take

• tile

• transpose

Supported Numpy Universal Functions (UFuncs)

To simplify the way we perform some math operations, we also support some important Numpy Universal Functions, also named ufunc. However, we had to simplify the implementation to get it running properly. So, the traditional kwargs passed to the ufunc, like out, where and others are ignored in our implementation.

The current supported ufuncs are:

• absolute (abs)

• add

• arccos

• arccosh

• arcsin

• arcsinh

• arctan

• arctanh

• cos

• cosh

• deg2rad

• degrees

• divide (true_divide)

• exp

• exp2

• expm1

• floor_divide

• hypot

• log

• log10

• log1p

• log2

• negative

• positive

• power (float_power)

• rad2deg

• radians

• remainder (mod)

• sin

• sinh

• square

• sqrt

• subtract

• tan

• tanh

All the units checking is automatically performed by astropop. In fact, most of these functions just wrap standart operations of QFloat, since this class perform the operations in a way very similar to the ufunc.

Trigonometric Math

For trigonometric functions, like sines, cosines and tangents, the code is able to check the dimensionality of the numbers before perform the operation. So, only dimensionless numbers are accepted, being dimensionless_unscaled or dimensionless_angles. Any number with pyshical dimension don’t make any sense inside trigonometric operations, which will raise an UnitsError.

To avoid an additional module containing trigonometric functions inside astropop, these operations are performed using the ufunc described earlier. The main difference here is the unit checking performed by astropop.

In [44]: qf = QFloat(30, 0.1, 'deg')

In [45]: np.cos(qf)
Out[45]: 
<QFloat at 0x7f754a16fe10>
0.8660+-0.0009

In [46]: np.sin(qf)
Out[46]: 
<QFloat at 0x7f754afb35d0>
0.500+-0.002

For trionometric (and hyperbolic) functions, like sin and sinh, only angle are accepted. So, only QFloat with degree or radians will not raise UnitsError. Also, all these functions will result in dimensionless_unscaled values.

For inverse trigonometric functions, like arcsin, the inverse happens. The input must be a dimensionless_unscaled QFloat, and output will be in units of radians.

In [47]: qf = QFloat(0.5, 0.01)

In [48]: np.arcsin(qf)
Out[48]: 
<QFloat at 0x7f754a319890>
0.52+-0.01 rad

Comparisons Notes

Comparing two numbers with units and uncertainties is an ambiguous thing. There are multiple ways to consider two numbers equal or different, or even greater or smaller. Due to this, we had to assume some conventions in the processing.

Equality

We consider two numbers equal if they have the same nominal and standard deviation values in the same unit. This means, they are exactly equal in everything, meaning a more programing-like approach. Like:

# 1.0+/-0.1 meters
In [49]: qf1 = QFloat(1.0, 0.1, 'm')

# same as above, but in cm
In [50]: qf2 = QFloat(100, 10, 'cm')

In [51]: qf1 == qf2
Out[51]: True

So, the simple fact that two numers have different error bars imply that they are different.

# 1.0+/-0.2 meters. Same number, with different error
In [52]: qf3 = QFloat(1.0, 0.2, 'm')

In [53]: qf1 == qf3
Out[53]: False

# 0.5+/-0.1 meters
In [54]: qf4 = QFloat(0.5, 0.1, 'm')

In [55]: qf1 == qf4
Out[55]: False

Of course, the different operator works in the exactly same way.

In [56]: qf1 != qf2
Out[56]: False

In [57]: qf1 != qf3
Out[57]: True

In [58]: qf1 != qf4
Out[58]: True

When comparing numbers with same dimension units, the code automatically converts if to compare. But, if incompatible units (different dimensions) are compared, they automatically are considered different. In physical terms, 1 second is different from 1 meter.

# Same nominal values of qf1, but in seconds
In [59]: qf5 = (1.0, 0.1, 's')

In [60]: qf1 == qf5
Out[60]: False

In [61]: qf1 != qf5
Out[61]: True

Equality considering errors

To check physical equality, which consider if the numbers are equal inside the error bars, we created the equal_within_errors method. In this method we assume two numbers (\(a\) and \(b\)) are equal if their different is smaller than the sum of the errors (\(\sigma_a\) and \(\sigma_b\)).

\[| a - b | <= \sigma_a + \sigma_b\]

In other words, these two numbers are equal if they intercept each other, considering error bars. Or, within the errors, they have ate least one value in common.

So, for a proper physical check of equalities, use equal_within_errors instead of == operator. For example:

In [62]: from astropop.math.physical import QFloat, equal_within_errors

In [63]: qf1 = QFloat(1.1, 0.1, 'm')

In [64]: qf2 = QFloat(1.15, 0.05, 'm')

In [65]: equal_within_errors(qf1, qf2)
Out[65]: True

Inequalities

Inequality handling is more ambiguous then equality to define. To avoid a complex API and keep the things in a coherent way, we perform greater, greater or equal, smaller and smaller or equal operations just comparing the nominal values of the numbers. For example:

In [66]: qf1 = QFloat(1.1, 0.1, 'm')

In [67]: qf2 = QFloat(1.15, 0.05, 'm')

In [68]: qf1 < qf2
Out[68]: True

In [69]: qf2 >= qf1
Out[69]: True

In [70]: qf2 < qf1
Out[70]: False

Note that errors are note being considered in operations. However, this operation perfmors full handling of physical units. So:

In [71]: qf1 = QFloat(1.0, 0.1, 'm')

In [72]: qf2 = QFloat(50, 10, 'cm')

In [73]: qf1 >= qf2
Out[73]: True

These comparisons can only be performed by same dimension measurements. If incompatible units are used, UnitsError is raised.

Physical Quantities API

astropop.math.physical Module

Math operations with uncertainties and units.

Simplified version of uncertainties python package with some units addings, in a much more free form.

Functions

unit_property(cls)	Add a unit property to a class.
qfloat(value[, uncertainty, unit])	Create a QFloat from the values.
equal_within_errors(qf1, qf2)	Check if two QFloats are equal within errors.

Classes

QFloat(value[, uncertainty, unit])	Storing float values with stddev uncertainties and units.
UnitsError	The base class for unit-specific exceptions.
UnitsError	The base class for unit-specific exceptions.