Skip to main content

Universal Ink Data Format

The Universal Ink Model is serialized as Protocol Buffers v3 message and wrapped within a RIFF container. The sections below describe the container and the protobuf serialization scheme.

RIFF Container

The Universal Ink Model (UIM) is serialized into a RIFF container, within the RIFF chunk UIM3. This chunk contains two sub-chunks, namely HEAD and DATA. The HEAD chunk contains the current version of the specification. The version is encoded using 3 bytes and is defined as follows: [major-version]-[minor-version]-[patch-version].

The ink model is serialized as Protocol Buffers message as described in section 8.2 Protocol Buffers Serialization Scheme and stored in the DATA chunk.

Logical Parts of Ink Model.

Figure 1: RIFF diagram

Ink Tree

The ink tree is built with a generic node structure. For building the tree the depth attribute reflects the depth within the tree. For serialization of the tree structure, the depth first pre-order tree serialization is applied, as illustrated in Figure 3.

Each node has an unique identifier id which is relevant for the semantic statements as an identifier for the subject. The index attribute links to the path or sensor data index. The groupBoundingBox is optional and assists with easier visual debugging or to highlight the relevant area for clickable options. Only when the type of the node is CHUNK the attributes chunkFromIndex and chunkToIndex are relevant for sub-indexing.

Hierarchy of groups

Figure 2: Depth-First tree order in red: F, B, A, D, C, E, G, I, H.

Data compression


The encoding process works as follows:

Step 1. (optional) Convert floating-point values to integer values using externally-specified decimal precision:

integerValue = floatValue * (10 ^ decimalPrecision);

Note: Step 1 is only applied if there the datatype is not an integer datatype (e.g., UINT32, UINT64, INT32, or INT64).

Step 2. Perform delta encoding on the converted values:

encodedValues[0] = integerValues[0];
for(i = 1; i < n; i++)
encodedValues[i] = integerValues[i] - integerValues[i - 1];

Note: In converting between floating-point values and fixed-point (integer) values, some precision can be lost. If precision is critical to application functionality, you must take this into consideration. A possible solution is to rely on IEEE 754: in this case, for every value to be converted, you should use the appropriate decimalPrecision and an approximation that remains unchanged during this conversion.


To restore the original values, the encoding process is reversed, as follows:

Step 1. Perform delta decoding to restore the integer values:

integerValues[0] = encodedValues[0];
for(i = 1; i < n; i++)
integerValues[i] = integerValues[i - 1] + encodedValues[i];

Step 2. (optional) Convert integer values to floating-point values using externally-specified decimal precision:

floatValue = integerValue / (10 ^ decimalPrecision)

Note: Step 2 is only applied if the datatype is not an integer datatype.

Protobuf Serialization Scheme


Definition of polygon prototype for brush.

coordXfloatrepeatedList of x coordinates of brush prototype.
coordYfloatrepeatedList of y coordinates of brush prototype.
coordZfloatrepeatedList of z coordinates of brush prototype [for 3D rendering].
indicesuint32repeatedList of indices of brush prototype [for 3D rendering].
shapeURIstringoptionalURI uniquely identifying shape's polygon.
sizefloatoptionalDefines prototype per path point size.


Brush descriptions, needed for ink rasterization.

vectorBrushesVectorBrushrepeatedList of defined vector brushes.
rasterBrushesRasterBrushrepeatedList of defined raster brushes.


List of data items.

sensorChannelIDstringoptionalReferencing InkSensorChannel via id.
valuessint32repeatedSample values delta encoded with provided precision.


Environment defines the all details regarding the input environment:

  • Operating System: name, version, etc.
  • Application: name, version, etc.
  • Geographic information: GPS coordinates.
idstringoptionalMD5 hash according to the MD5 hash-based Unique Identifier Generation Scheme.
propertiesPropertyrepeatedEnvironment properties, such as OS name, OS, version, etc.


Float value wrapper to avoid cross-platform zero value issues.

valuefloatoptionalFloat value wrapped within the message.


Contains all ink data.

strokesStrokerepeatedData holder for strokes.


Definition of input provider types.

idstringoptionalMD5 hash according to the MD5 hash-based Unique Identifier Generation Scheme.
typeInkInputProviderTypeoptionalType of used hardware - PEN, TOUCH, MOUSE, or controller.
propertiesPropertyrepeatedInkInputProvider properties, such as PenID.


Ink Object as base structure to capture all relevant data.

  • Sensor data recorded from the input sensor
  • Brush configurations defining the different brush types used
  • Ink data with the visual geometry of the ink strokes
  • Ink tree structuring the ink data
  • Meta-data describing the author, document, etc.
  • Knowledge graph with semantics
inputDataInputDataoptionalData holder for ink input related data.
inkDataInkDataoptionalData holder for ink related ink data.
brushesBrushesoptionalRasterization tools.
inkTreeNoderepeatedThe ink-tree: this is the ink-centric data model.
viewsViewrepeatedA list of views of the ink-tree based on the knowledge graph.
knowledgeGraphTripleStoreoptionalThe Knowledge Graph.
transformMatrix4optionalTransform is an affine transformation matrix which is applied to all paths.
propertiesPropertyrepeatedDocument related properties, such as DOCUMENT, AUTHOR, LAST_MODIFIED, etc.


Capturing context of the ink device with reference to the Environment and the SensorContext.

idstringoptionalMD5 hash according to the MD5 hash-based Unique Identifier Generation Scheme.
environmentIDstringoptionalReference to the Environment.
sensorContextIDstringoptionalReference to the SensorContext.


Container structure for context and input defines.

inputContextsInputContextrepeatedList of input contexts.
inkInputProvidersInkInputProviderrepeatedList of input providers.
inputDevicesInputDevicerepeatedList of input devices.
environmentsEnvironmentrepeatedList of environment setups.
sensorContextsSensorContextrepeatedList of sensor contexts.


Input Data bundles the input.

inputContextDataInputContextDataoptionalInput context data.
sensorDataSensorDatarepeatedData holder for sensor data.


InputDevice with its properties.

The properties can contain:

  • Communication Protocol: USB, BTC, BLE, SPP, WIFI
  • Communication id: version-WILL3-VID, PID; MAC; UID; COM_PORT
  • Device Name: Wacom Intuos Pro M, Apple iPad 8, Samsung GalaxyTab 10
  • Serial number
  • Firmware Version (MCU)
  • Secondary Firmware Version (BT, WIFI) - different modules provides version for itself
  • Sensor size
idstringoptionalMD5 hash according to the MD5 hash-based Unique Identifier Generation Scheme.
propertiesPropertyrepeatedInputDevice properties.


Representation of a 4x4 affine matrix

| m00 m01 m02 m03 | | m10 m11 m12 m13 | | m20 m21 m22 m23 | | m30 m31 m32 m33 |



The node message used for the definition of tree-structures.

idstringoptionalNode unique identifier
depthuint32optionalDepth in hierarchy structure
indexuint32optionalIndex in InkData.paths or in InkData.sensorData data holder
typeNodeTypeoptional(PATH & SENSOR_DATA nodes only)Node type
groupBoundingBoxRectangleoptionalBounding box (Group nodes only)
chunkFromIndexuint32optionalChunk beginning (CHUNK node only)
chunkToIndexuint32optionalChunk ending (CHUNK node only)


Pipeline properties which not varies. Variable ones are serialized via Stroke message.

sizeFloat32optionalBrush size.
redFloat32optionalColor value [0, 1] for red channel.
greenFloat32optionalColor value [0, 1] for green channel.
blueFloat32optionalColor value [0, 1] for blue channel.
alphaFloat32optionalColor value [0, 1] for alpha channel.
rotationFloat32optionalBrush rotation z.
scaleXFloat32optionalBrush scale x.
scaleYFloat32optionalBrush scale y.
scaleZFloat32optionalBrush scale z [for 3D rendering].
offsetXFloat32optionalBrush offset x.
offsetYFloat32optionalBrush offset y.
offsetZFloat32optionalBrush offset z [for 3D rendering].


Encodes a single property

namestringoptionalName of the property.
valuestringoptionalValue of property.


A brush defined with raster images

namestringoptionalthe brush descriptor.
spacingfloatoptionalDistance between neighbour particles.
scatteringfloatoptionalThe scattering along the curve normal.
rotationModeRotationModeoptionalThe particle rotation mode of the brush.
shapeTexturebytesrepeatedList of png images that contains the shape texture.
shapeTextureURIstringrepeatedList of URIs associated to the shape texture.
fillTexturebytesoptionalList of png image that contains the fill texture.
fillTextureURIstringoptionalURI identifying the fillTexture.
fillWidthfloatoptionalWidth of the fill tile.
fillHeightfloatoptionalHeight of the fill tile.
randomizeFillbooloptionalSpecifies whether the fill texture is randomly displaced.
blendModeBlendModeoptionalThe applied blend mode.


Representation of rectangle

xfloatoptionalLower-left x-coordinate point of rectangle.
yfloatoptionalLower-left y-coordinate point of rectangle.
widthfloatoptionalWidth of rectangle.
heightfloatoptionalHeight of rectangle.


A semantic triple, or simply triple, is the atomic data entity data model. As its name indicates, a triple is a set of three entities that codifies a statement about semantic data in the form of subject predicate object expressions.



Information about the sensor channel.

Supported channel types:

  • will:///input/3.0/channel/X
  • will:///input/3.0/channel/Y
  • will:///input/3.0/channel/Z
  • will:///input/3.0/channel/Timestamp
  • will:///input/3.0/channel/Pressure
  • will:///input/3.0/channel/RadiusX
  • will:///input/3.0/channel/RadiusY
  • will:///input/3.0/channel/Altitude
  • will:///input/3.0/channel/Azimuth
  • will:///input/3.0/channel/Rotation
idstringoptionalMD5 hash according to the MD5 hash-based Unique Identifier Generation Scheme.
typestringoptionalURI which describes uniquely this channel. Supported types are:
metricInkSensorMetricTypeoptionalUser defined channels which introduce custom meaning should be provided from user side.Indicates metric used in calculating the resolution for the data item.
resolutiondoubleoptionalIs a decimal number giving the number of data item increments.
minfloatoptionalPer physical unit., e.g. if the physical unit is in m and device units./ Resolution is 100000, then the value 150 would be 0.0015 m.Minimal value of the channel
maxfloatoptionalMaximal value of the channel
precisionuint32optionalPrecision of integer encoding, needed for encoded float values


The SensorChannelsContext groups sensor channels that are producing data with the same sampling rate.

idstringoptionalMD5 hash according to the MD5 hash-based Unique Identifier Generation Scheme.
channelsSensorChannelrepeatedChannels in this group provides aligned values in the same time.
samplingRateHintUint32optionalHint for the intended sampling rate of the channel [Optional].
latencyUint32optionalLatency measure in milliseconds [Optional].
inkInputProviderIDstringoptionalReference to the InkInputProvider.
inputDeviceIDstringoptionalReference to the InputDevice.


Each input device has a SensorContext describing the available sensors of a device. One file can contain Ink data from two device of the same type with a shared context.

idstringoptionalMD5 hash according to the MD5 hash-based Unique Identifier Generation Scheme.
sensorChannelsContextSensorChannelsContextrepeatedGrouping context for device sensor channels.


SensorData is the central data structure for storing raw data from the ink sensor.

inputContextIDstringoptionalReferences the InputContext via id.
stateInkStateoptionalState of the ink.
timestampuint64optionalTimestamp for first sample of the stroke, measured in milliseconds.
dataChannelsChannelDatarepeated(time when first X & Y are reported). Data channels encoding the data list.


Visual path structure. Stroke in its simplest form contains one or more Segments (Catmull-Rom), which includes a series of positional points, Positions. In addition to those properties, digital ink strokes include start parameter and end parameter as Stroke entity's properties.

idstringoptionalInternal id.
startParameterfloatoptionalSpline start parameter [0, 1].
endParameterfloatoptionalSpline end parameter [0, 1].
splineXfloatrepeatedSpline values for x.
splineYfloatrepeatedSpline values for y.
splineZfloatrepeatedSpline values for z [for 3D rendering].
redfloatrepeatedColor values [0, 1] for red channel.
greenfloatrepeatedColor values [0, 1] for green channel.
bluefloatrepeatedColor values [0, 1] for blue channel.
alphafloatrepeatedColor values [0, 1] for alpha channel.
sizefloatrepeatedBrush size.
rotationfloatrepeatedBrush rotation z.
scaleXfloatrepeatedBrush scale x.
scaleYfloatrepeatedBrush scale y.
scaleZfloatrepeatedBrush scale z [for 3D rendering].
offsetXfloatrepeatedBrush offset x.
offsetYfloatrepeatedBrush offset y.
offsetZfloatrepeatedBrush offset z [for 3D rendering].
sensorDataOffsetuint32optionalIndex of points mapping between raw and processed paths.
sensorDataIDstringoptionalReference sensor data.
sensorDataMappinguint32repeatedExplicit mapping between indices of Stroke and SensorData, used when input rate is very high and provides unwanted points.
styleStyleoptionalStyle that is applied to the path.


A style message, used for Stroke customization.

propertiesPathPointPropertiesoptionalStatic values of properties which do not exist per per path point
brushURIstringoptionalReference to Brush used for stroke rasterization.
particlesRandomSeeduint32optionalParticles random seed, required for particle strokes.
renderModeURIstringoptionalDefines additional information about stroke visualisation, such as ERASER.


Encapsulates a list of TripleGroups.

statementsSemanticTriplerepeatedList of semantic triples.


Unsigned integer value wrapper to avoid cross-platform zero value issues.

valueuint32optionalUnsigned integer value wrapped within the message.


A brush defined with a polygon (polyhedron for 3D).

namestringoptionalBrush descriptor.
prototypeBrushPrototyperepeatedPolygon prototype for the brush.
spacingfloatoptionalSpacing value.


A message representing a view of the ink-tree, which is serialized in form of another tree.

namestringoptionalView descriptor URI (unique in the model scope).
treeNoderepeatedThe tree structure of the current view.


The blend mode enum lists the different blend modes which can be applied to raster brushes.

SOURCE_OVER0This is the default setting and draws new shapes on top of the existing canvas content. Also known as NORMAL.
DESTINATION_OVER1New shapes are drawn behind the existing canvas content. Also known as NORMAL_REVERSE.
DESTINATION_OUT2The existing content is kept where it doesn't overlap the new shape. Also known as ERASE.
LIGHTER3Where both shapes overlap the color is determined by adding color values. Also known as ADD.
COPY4Only the new shape is shown. Also known as NONE.
MIN5The result is the minimum of both color. The result is a darker color.
MAX6The result is the maximum of both color. The result is a lighter color.


Defining the different types of input.

PEN0Stylus, smart pens, pen displays, signature capturing device, etc.
TOUCH1Touch controller input: Finger or passive stylus.
MOUSE2Mouse device.
CONTROLLER33-DOF or 6-DOF input devices.


Supported metric types

LENGTH0Underlying si unit is meter.
TIME1Underlying si unit is second.
FORCE2Underlying si unit is newton.
ANGLE3Underlying si unit is radian.
NORMALIZED4Percentage, expressed as a fraction (1.0 = 100%) relative to max-min.
LOGICAL5Logical value, true / false values.
DIMENSIONLESS6No metric defined for sensor.


The ink device state defines the state of the Ink device. WILL 3 supports different modes:

  • Writing on a plane
  • Hovering above a surface
  • Moving in air (VR/AR/MR) interaction
  • Only hovering in the air
PLANE0Ink device is writing on a surface.
HOVERING1Hovering over a surface.
IN_VOLUME2Using the ink device in the air, with active inking.
VOLUME_HOVERING3Moving the pen in the air with disabled inking.


Defines the node type.

PATH_GROUP0PATH_GROUP is hierarchically combining stroke nodes.
SENSOR_DATA_GROUP1SENSOR_DATA_GROUP is hierarchically combining sensor data nodes.
PATH2PATH is a node type where nodes directly point to a stroke.
SENSOR_DATA3SENSOR_DATA is a node type where nodes directly point to a sensor data sample sequence.


The rotation parameter specifies the rotation of the particle.

NONE0Indicates that the shape will not be rotated.
RANDOM1Indicates that the shape will be rotated randomly before it renders.
TRAJECTORY2Indicates that the shape will be rotated to match the path trajectory.

Scalar Value Types

.proto TypeNotesC++ TypeJava TypePython Type
int32Uses variable-length encoding. Inefficient for encoding negative numbers - if your field is likely to have negative values, use sint32 instead.int32intint
int64Uses variable-length encoding. Inefficient for encoding negative numbers - if your field is likely to have negative values, use sint64 instead.int64longint/long
uint32Uses variable-length encoding.uint32intint/long
uint64Uses variable-length encoding.uint64longint/long
sint32Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s.int32intint
sint64Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s.int64longint/long
fixed32Always four bytes. More efficient than uint32 if values are often greater than 2^28.uint32intint
fixed64Always eight bytes. More efficient than uint64 if values are often greater than 2^56.uint64longint/long
sfixed32Always four bytes.int32intint
sfixed64Always eight bytes.int64longint/long
stringA string must always contain UTF-8 encoded or 7-bit ASCII text.stringStringstr/unicode
bytesMay contain any arbitrary sequence of bytes.stringByteStringstr