Skip to main content

Class: FixedValuePrecisionCalculator

Serialization.PrecisionDetection.FixedValuePrecisionCalculator(precisionopt, autoAdjustOnIntegerOverflowopt)

A precision calculator based on the error introduced after rounding

Constructor

new FixedValuePrecisionCalculator(precisionopt, autoAdjustOnIntegerOverflowopt)

Parameters:
NameTypeAttributesDefaultDescription
precisionint<optional>2Fixed precision value
autoAdjustOnIntegerOverflowboolean<optional>falseWhen is true adjust precision to skip overflow

Implements:

Methods

calculatePrecision(data, property) → {int}

Calculates an estimated precision value based on the error introduced after rounding

Parameters:
NameTypeDescription
dataFloat32ArrayThe data to be used for the calculation
propertyPathBuilding.PathPoint.PropertyGeometric/appearance property, which corresponds to the given sequence of path point data values

Implements:

Returns:

A decimal precision value

Type

int

Home