墨水序列化

Ink Serialization模块提供墨水内容及相关元数据的编码和解码算法。这些算法经过优化，能够很好地与Universal Ink Model适应.

算法支持以下特性：

快速编解码
紧凑文件大小
不同操作系统和设备间的可移植性
可嵌入不同容器格式的二进制表示法

要阅读关于编码方案的更多信息，请单击here.

处理模型

The following illustrates how to manage the data used within the Universal Ink Model。在墨水模型中，如果使用案例需要相关信息，则需要管理多个数据库 - 以及墨水树、视图和知识图谱。

Logical Parts of Ink Model.

Ink Model

第一步是创建InkModel.

Kotlin
C#
JavaScript

inkModel = InkModel()

public class Serializer
{
  ...
public void Init()
 {
     InkDocument = new InkModel();
     InkDocument.InkTree.Root = new StrokeGroupNode(Identifier.FromNewGuid());
 }

class DataModel {
	constructor() {
		this.inkModel = new InkModel();
		this.repository = new DataRepository();
		this.manipulationsContext = new SpatialContext();
	}
...

输入上下文数据库

输入上下文环境描述如何捕获传感器数据。该信息需要由应用程序添加，但 -- 根据应用程序的使用案例 -- 也可能不需要。传感器数据的存储也是可选的。

下列章节提供了如何定义上下文环境的一些实例。

环境

生成传感器数据（操作系统等）的environment可按照下列方式来定义：

Kotlin
C#

import com.wacom.ink.format.input.*
...

// Initialize environment
environment = Environment()
environment.putProperty("os.name", "android")
environment.putProperty("os.version.name", Build.VERSION.CODENAME)
environment.putProperty("os.version.code", Build.VERSION.SDK_INT.toString())
environment.putProperty("os.version.incremental", Build.VERSION.INCREMENTAL)
environment.putProperty("os.version.release", Build.VERSION.RELEASE)
environment.putProperty("wacom.ink.sdk.name", activity.getString(R.string.sdk_name))
environment.putProperty("wacom.ink.sdk.version", activity.getString(R.string.sdk_version))

// Init environment
mEnvironment.Properties["os.name"] = m_eas.OperatingSystem;
mEnvironment.Properties["os.version.code"] = System.Environment.OSVersion.Version.ToString();
mEnvironment.Seal();

InkInputProvider

ink input provider 表示通用输入数据源。它标识数据的生成方式（使用触摸输入、鼠标、触控笔、硬件控制器等）。对于每个平台和设备，应用程序必须确定它使用哪种输入设备，如何在墨水模型中序列化。

Kotlin
C#
JavaScript

import com.wacom.ink.format.input.*
...
val toolType = when (event.getToolType(0)) {
    MotionEvent.TOOL_TYPE_STYLUS -> InkInputType.PEN
    MotionEvent.TOOL_TYPE_FINGER -> InkInputType.TOUCH
    MotionEvent.TOOL_TYPE_MOUSE -> InkInputType.MOUSE
    else -> InkInputType.PEN
}

val provider = InkInputProvider(toolType)

TBD

TBD

输入设备

input device 用于生成传感器数据。根据所使用的平台，可能会有多种输入设备可用。一些平板电脑提供触控笔，或者可作为附件提供。下列属性可以根据需要进行收集，例如，由平台自身。

Kotlin
C#
JavaScript

import com.wacom.ink.format.input.*
...

// Initialize InputDevice
inputDevice = InputDevice()
inputDevice.putProperty("dev.id", Build.ID)
inputDevice.putProperty("dev.manufacturer", Build.MANUFACTURER)
inputDevice.putProperty("dev.brand", Build.BRAND)
inputDevice.putProperty("dev.model", Build.MODEL)
inputDevice.putProperty("dev.board", Build.BOARD)
inputDevice.putProperty("dev.hardware", Build.HARDWARE)
inputDevice.putProperty("dev.codename", Build.DEVICE)
inputDevice.putProperty("dev.display", Build.DISPLAY)
...

{
    InputDevice inputDevice = new InputDevice();
    inputDevice.Properties["dev.name"] = System.Environment.MachineName;
    inputDevice.Seal();

    Identifier inputDeviceId = inputDevice.Id;
    bool res = InkDocument.InputConfiguration.Devices.Any((device) => device.Id == inputDeviceId);

    if (!res)
    {
        InkDocument.InputConfiguration.Devices.Add(inputDevice);
    }

    return inputDevice;
}

/**
 * Creates default input device instance, based on system information
 *
 * @param {Properties} envProps User defined env properties, like app.id for example
 * @return {InkInput.InputDevice} default instance
 */
static async createInstance(envProps) {
  let device = new this(...Array.from(arguments).slice(1));

  if (typeof sysInfo == "undefined")
    device.props["dev.graphics.resolution"] = `${screen.width}x${screen.height}`;
  else {
    let system = await sysInfo.system();
    let cpu = await sysInfo.cpu();
    let graphics = await sysInfo.graphics();

    let display = graphics.displays.filter(d => d.main)[0];
    let adapter = graphics.controllers[0];

    device.props["dev.id"] = system.uuid.toLowerCase();
    device.props["dev.manufacturer"] = system.manufacturer;
    device.props["dev.model"] = system.model;

    device.props["dev.cpu"] = `${cpu.manufacturer} ${cpu.brand} ${cpu.speed} - ${cpu.cores} core(s)`;

    device.props["dev.graphics.display"] = `${display.model} ${display.currentResX}x${display.currentResY} (${display.pixeldepth} bit)`;
    device.props["dev.graphics.adapter"] = `${adapter.model} ${adapter.vram} GB`;
  }

  device.environment = await Environment.createInstance(envProps);

  return device;
}

...

let device = await InputDevice.createInstance({"app.id": "will3-sdk-for-ink-web-demo", "app.version": "1.0.0"});

SensorContext

sensor context定义了捕获数字墨水输入的传感器通道上下文环境的唯一组合。对于在机器学习上下文环境中使用的传感器数据，如果需要标准化，该信息很有用。

Kotlin
C#

// Define which channels are used for sensor data collection
var inkSensorTypes = listOf(InkSensorType.X, InkSensorType.Y, InkSensorType.TIMESTAMP)
...
// Register the channels which are used for sensor data collection
val channels = registerChannels(inkSensorTypeUris)

// Define the context for the sensor channels
val sensorChannelsContext = SensorChannelsContext(provider.id, inputDevice.id, channels)
val sensorContext = SensorContext()
sensorContext.addSensorChannelsContext(sensorChannelsContext)

// Maintain a map of contexts by id
sensorContexts[sensorContext.id] = sensorContext

// Input context is combining environment and sensor context  
val inputContext = InputContext(environment.id, sensorContext.id)

// Maintain a map of input context
inputContexts[inputContext.id] = inputContext
inputProviderToInputContextMapping[provider.id] = inputContext.id

...
private fun registerChannels(inkSensorTypeUris: List<String>): MutableList<SensorChannel> {
    val precision = 2

    val channels = mutableListOf<SensorChannel>()

    val dimensions = getScreenDimensions()

    for (type in inkSensorTypeUris) {
        val channel = when (type) {
            InkSensorType.X -> SensorChannel(
                InkSensorType.X,
                InkSensorMetricType.LENGTH,
                ScalarUnit.INCH,
                0.0f,
                dimensions.x,
                precision
            )
            InkSensorType.Y -> SensorChannel(
                InkSensorType.Y,
                InkSensorMetricType.LENGTH,
                ScalarUnit.INCH,
                0.0f,
                dimensions.y,
                precision
            )
            InkSensorType.Z -> SensorChannel(
                InkSensorType.Z,
                InkSensorMetricType.LENGTH,
                ScalarUnit.DIP,
                0.0f,
                0.0f,
                precision
            )
            InkSensorType.TIMESTAMP -> SensorChannel(
                InkSensorType.TIMESTAMP,
                InkSensorMetricType.TIME,
                ScalarUnit.MILLISECOND,
                0.0f,
                0.0f,
                precision
            )
            InkSensorType.PRESSURE -> SensorChannel(
                InkSensorType.PRESSURE,
                InkSensorMetricType.NORMALIZED,
                ScalarUnit.NORMALIZED,
                0.0f,
                1.0f,
                precision
            )
            InkSensorType.RADIUS_X -> SensorChannel(
                InkSensorType.RADIUS_X,
                InkSensorMetricType.LENGTH,
                ScalarUnit.DIP,
                0.0f,
                0.0f,
                precision
            )
            InkSensorType.RADIUS_Y -> SensorChannel(
                InkSensorType.RADIUS_Y,
                InkSensorMetricType.LENGTH,
                ScalarUnit.DIP,
                0.0f,
                0.0f,
                precision
            )
            InkSensorType.ALTITUDE -> SensorChannel(
                InkSensorType.ALTITUDE,
                InkSensorMetricType.ANGLE,
                ScalarUnit.RADIAN,
                0.0f,
                (Math.PI/2).toFloat(),
                precision
            )
            InkSensorType.AZIMUTH -> SensorChannel(
                InkSensorType.AZIMUTH,
                InkSensorMetricType.ANGLE,
                ScalarUnit.RADIAN,
                -(Math.PI/2).toFloat(),
                (Math.PI/2).toFloat(),
                precision
            )
            InkSensorType.ROTATION -> SensorChannel(
                InkSensorType.ROTATION,
                InkSensorMetricType.ANGLE,
                ScalarUnit.RADIAN,
                0.0f,
                0.0f,
                precision
            )
            else -> {
                throw Exception("Unknown channel type.")
            }
        }
        channels.add(channel)
    }
    return channels
}

// Init sensor channels contexts
SensorChannelsContext mouseSensorChannelsContext = SensorChannelsContext.CreateDefault(mouseInputProvider, m_currentInputDevice); //new SensorChannelsContext(mouseInputProvider, m_currentInputDevice, m_sensorChannels);
SensorChannelsContext touchSensorChannelsContext = SensorChannelsContext.CreateDefault(touchInputProvider, m_currentInputDevice); //new SensorChannelsContext(touchInputProvider, m_currentInputDevice, m_sensorChannels);
SensorChannelsContext penSensorChannelsContext = SensorChannelsContext.CreateDefault(penInputProvider, m_currentInputDevice); //new SensorChannelsContext(penInputProvider, m_currentInputDevice, m_sensorChannels);

// Cache sensor channels contexts
m_sensorChannelsContexts.Add(mouseSensorChannelsContext.Id, mouseSensorChannelsContext);
m_sensorChannelsContexts.Add(touchSensorChannelsContext.Id, touchSensorChannelsContext);
m_sensorChannelsContexts.Add(penSensorChannelsContext.Id, penSensorChannelsContext);

SensorData 版本库

要追踪传感器数据，在所有平台上都知道InkInputProvider很重要，便可从事件中推测出来。而且，渲染特性也各不相同，例如，触摸和触控笔输入提供者，它们的渲染特性就不同。

Kotlin
C#
JavaScript

...
val touchChannels = listOf(InkSensorType.X, InkSensorType.Y, InkSensorType.TIMESTAMP)
val penChannels = listOf(InkSensorType.X, InkSensorType.Y, InkSensorType.TIMESTAMP,
                         InkSensorType.PRESSURE, InkSensorType.ALTITUDE, InkSensorType.AZIMUTH)
...


fun createSensorData(event: MotionEvent): Pair<SensorData, List<SensorChannel>> {
    // MotionEvent tool type helps to decide on the appropriate ink input provider
    val toolType = when (event.getToolType(0)) {
        MotionEvent.TOOL_TYPE_STYLUS -> InkInputType.PEN
        MotionEvent.TOOL_TYPE_FINGER -> InkInputType.TOUCH
        MotionEvent.TOOL_TYPE_MOUSE -> InkInputType.MOUSE
        else -> InkInputType.PEN
    }

    var provider: InkInputProvider? = null
    // First, check if exist an input provider of the desired type
    for ((_, existingProvider) in inputProviders) {
        existingProvider.type == toolType
        provider = existingProvider
        break
    }

    if (provider == null) {
        provider = InkInputProvider(toolType)

        // It is possible to add custom define properties to the input provider
        // for example PenID in case exists
        if (toolType == InkInputType.PEN) {
            provider.putProperty("penType", "s-pen") // Assuming using Samsung S Pen
        }
    }

    var inputContextId = if (!inputProviders.containsKey(provider.id)) {
        inputProviders[provider.id] = provider

        // Build the list of channels
        val channels = registerChannels(if (toolType == InkInputType.PEN) penChannels else touchChannels)
        channelsForInput.put(provider.id, channels)

        val sensorChannelsContext = SensorChannelsContext(
          provider.id,       // Reference to input input provider
          inputDevice.id,    // Reference to input device
          channels)          // Channels for the registered channels

        val sensorContext = SensorContext()
        sensorContext.addSensorChannelsContext(sensorChannelsContext)

        sensorContexts[sensorContext.id] = sensorContext

        val inputContext = InputContext(
          environment.id,      // Reference to environment
          sensorContext.id)    // Reference to sensor context

        inputContexts[inputContext.id] = inputContext

        inputProviderToInputContextMapping[provider.id] = inputContext.id

        inputContext.id
    } else {
        val provider = inputProviders[provider.id]!!
        inputProviderToInputContextMapping[provider.id]!!
    }
    var channelList: List<SensorChannel>? = null
    for ((id, channels) in channelsForInput) {
        if (provider.id == id) {
            channelList = channels
        }
    }

    if (channelList == null) {
        channelList = listOf() // empty list to avoid null pointer exceptions
    }
    // Create the SensorData object with a new unique ID and its input context reference id
    return Pair<SensorData,
    List<SensorChannel>>(SensorData(UUID.randomUUID().toString(), inputContextId, InkState.PLANE),
    channelList)
}


override fun onEvent(pointerData: PointerData, inkToolType: InkInputType) {
    when (pointerData.phase) {
        // Set the appropriate input provider
        Phase.BEGIN -> {
            if ((inputProvider.type != inkToolType)) {
                inputProvider = InkInputProvider(inkToolType)
            }
        }
        // Adding sensor data
        Phase.UPDATE, Phase.END -> {
            sensorData.add(sensorChannels[InkSensorType.X]!!, pointerData.x)
            sensorData.add(sensorChannels[InkSensorType.Y]!!, pointerData.y)
            sensorData.addTimestamp(sensorChannels[InkSensorType.TIMESTAMP]!!,
                                    pointerData.timestamp)
        }
    }
}
...
}

public Identifier AddSensorData(PointerDeviceType deviceType, List<PointerData> pointerDataList)
{
    Identifier inputContextId = m_deviceTypeMap[deviceType];
    InputContext inputContext = m_inputContexts[inputContextId];
    SensorContext sensorContext = m_sensorContexts[inputContext.SensorContextId];

    // Create sensor data using the input context
    SensorData sensorData = new SensorData(
        Identifier.FromNewGuid(),
        inputContext.Id,
        InkState.Plane);

    PopulateSensorData(sensorData, sensorContext, pointerDataList);

    m_sensorDataMap.TryAdd(sensorData.Id, sensorData);

    return sensorData.Id;
}
...
/// <summary>
      /// Make the current stroke permanent
      /// </summary>
      /// <remarks>Copies the output of the render pipeline from InkBuilder to dry strokes</remarks>
      public override void StoreCurrentStroke(PointerDeviceType deviceType)
      {
          var allData = RasterInkBuilder.SplineInterpolator.AllData;
          var points = new List<float>();

          if (allData != null)
          {
              for (int i = 0; i < allData.Count; i++)
              {
                  points.Add(allData[i]);
              }

              if (points.Count > 0)
              {
                  var dryStroke = new RasterInkStroke(RasterInkBuilder,
                      deviceType,
                      points,
                      m_startRandomSeed,
                      CreateSerializationBrush($"will://examples/brushes/{Guid.NewGuid().ToString()}"),
                      mStrokeConstants.Clone(),
                      mSerializer.AddSensorData(deviceType, InkBuilder.GetPointerDataList()));
                  m_dryStrokes.Add(dryStroke);
              }
          }
      }

add(stroke) {
  this.manipulationsContext.add(stroke);
  return this.inkModel.addPath(stroke);
}

...

begin(sensorPoint) {
  if (this.forward) return this.inkCanvasRaster.begin(sensorPoint);

  this.reset(sensorPoint);

  this.builder.add(sensorPoint);
  this.builder.build();
}

move(sensorPoint) {
  if (this.forward) return this.inkCanvasRaster.move(sensorPoint);

  if (app.downsampling && this.requested) {
    this.builder.ignore(sensorPoint);
    return;
  }

  this.builder.add(sensorPoint);

  if (!this.requested) {
    this.requested = true;

    this.builder.build();

    requestAnimationFrame(() => (this.requested = false));
  }
}

end(sensorPoint) {
  if (this.forward) return this.inkCanvasRaster.end(sensorPoint);

  this.builder.add(sensorPoint);
  this.builder.build();
}

Stroke 版本库

rendering pipeline的视觉输出也必须保存，它是通用墨水模型的关键部分。在此，需要添加下列结构：

Stroke - 墨水笔划的几何结构
Style - 笔划渲染的定制参数
StrokeNode - 墨水树的逻辑节点

Kotlin
C#
JavaScript

import com.wacom.ink.format.rendering.PathPointProperties
import com.wacom.ink.format.rendering.Style
import com.wacom.ink.format.tree.data.Stroke
import com.wacom.ink.format.tree.nodes.StrokeNode
import com.wacom.ink.model.IdentifiableImpl

...
fun surfaceTouch(event: MotionEvent) {
  ...
    if ((pointerData.phase == Phase.END) &&
        (rasterInkBuilder.splineProducer.allData != null)) {
        addStroke(event)
    }
   ...
}

...
private fun addStroke(event: MotionEvent) {
    // Adding the style
    val style = Style(
        rasterTool.uri(),              // Style URI
        1,                             // Particle random seed
        props = PathPointProperties(   // Coloring path properties
            red = defaults.red,       
            green = defaults.green,
            blue = defaults.blue,
            alpha = defaults.alpha
        ),
        renderModeUri = rasterTool.getBlendMode().name
    )
    // Adding stroke to the Stroke Repository
    val path = Stroke(
        IdentifiableImpl.generateUUID(),                    // Generated UUID
        rasterInkBuilder.splineProducer.allData!!.copy(),   // Spline
        style                                               // Style
    )
    // Adding a node to the Ink tree
    val node = StrokeNode(IdentifiableImpl.generateUUID(),
                          path)
    ...
    strokeNodeList.add(node)
}

private void EncodeStrokeCommon(Identifier id, Spline spline , PathPointLayout layout, Identifier sensorDataId, Style style)
{
    Stroke stroke = new Stroke(
        id,
        spline.Clone(),
        style,
        layout,
        sensorDataId);

    StrokeNode strokeNode = new StrokeNode(Identifier.FromNewGuid(), stroke);
    InkDocument.InkTree.Root.Add(strokeNode);

    if (sensorDataId != Identifier.Empty)
    {
        SensorData sensorData = m_sensorDataMap[sensorDataId];

        AddSensorDataToModel(sensorData);
    }
}

draw(pathPart) {
  this.drawPath(pathPart);

  if (pathPart.phase == InkBuilder.Phase.END) {
    if (this.strokeRenderer) {
      let stroke = this.strokeRenderer.toStroke(this.builder);
      this.dataModel.add(stroke)
    }
  }
}

Brush 版本库

由于需要确保墨水笔划可在不同平台或应用程序上进行渲染，每种刷子都必须是嵌入式的。

Kotlin
C#

fun setTool(view: View, tool: Tool) {
    drawingTool = tool

    if (drawingTool is VectorTool) {
        vectorDrawingView.setTool(drawingTool as VectorTool)
    } else {
        val dt = drawingTool as RasterTool
        val brush = dt.brush
        if (inkModel.brushRepository.getBrush(brush.name) == null) {
            // Adding a raster brush if it is not within the repository yet
            inkModel.brushRepository.addRasterBrush(brush as RasterBrush)
        }
        rasterDrawingSurface.setTool(dt)
    }
    highlightTool(view)
}

private void AddRasterBrushToInkDoc(PointerDeviceType deviceType, RasterBrush rasterBrush,
                                    Style rasterStyle, StrokeConstants strokeConstants,
                                    uint startRandomSeed)
{
    rasterStyle.RenderModeUri = $"will3://rendering//{deviceType.ToString()}";

    if (!InkDocument.Brushes.TryGetBrush(rasterBrush.Name, out Brush foundBrush))
    {
        InkDocument.Brushes.AddRasterBrush(rasterBrush);
    }
}

private void AddVectorBrushToInkDoc(string pointerDeviceType, Wacom.Ink.Serialization.Model.VectorBrush vectorBrush, Style style)
{
    style.RenderModeUri = $"will3://rendering//{pointerDeviceType}";

    if (!InkDocument.Brushes.TryGetBrush(vectorBrush.Name, out Brush foundBrush))
    {
        InkDocument.Brushes.AddVectorBrush(vectorBrush);
    }
}

暂留

为确保墨水模型的一致性以及可跨平台分享，需要将其编码到数据流中。

编码

模型将使用二进制表达式表面，更多细节请参见 here.

Kotlin
C#
JavaScript

try {
    val file = File(filePath)
    fileOutputStream = FileOutputStream(file)
    // Serialize encoded ink model to an output stream
    Will3Codec.encode(inkModel, fileOutputStream)
} catch (e: Exception){
    e.printStackTrace()
} finally {
    fileOutputStream?.close()
}

StorageFile file = await savePicker.PickSaveFileAsync();
if (file != null)
{
    // Prevent updates to the remote version of the file until we finish making changes and call CompleteUpdatesAsync.
    CachedFileManager.DeferUpdates(file);
    // Write to file
    await FileIO.WriteBytesAsync(file, Will3Codec.Encode(m_renderer.StrokeHandler.Serialize()));
    // Let Windows know that we're finished changing the file so the other app can update the remote version of the file.
    // Completing updates may require Windows to ask for user input.
    FileUpdateStatus status = await CachedFileManager.CompleteUpdatesAsync(file);

    if (status != FileUpdateStatus.Complete)
    {
        MessageDialog md = new MessageDialog("File could not be saved!", "Error saving file");
        await md.ShowAsync();
    }
}

 async encode() {
   return await this.codec.encodeInkModel(this.dataModel.inkModel);
 }

 ...

 async save() {
   let buffer = await this.encode();
   fsx.saveAs(buffer, "ink.uim", "application/vnd.wacom-ink.model");
 }

解码

为解码数据流，需要将编码后的数据传送到解码器中。由于数据格式的版本号编码在RIFF标题中，将自动选择合适的内容解析器版本。

Kotlin
C#
JavaScript

// Loading file stream from path
val fileInputStream = FileInputStream(File(path))
// Read bytes array with UIM content
val bytes = fileInputStream.readBytes()
// Using the codec decoder to de-serialize the InkModel
val inkModel = Will3Codec.decode(bytes)

StorageFile file = await picker.PickSingleFileAsync();
if (file != null)
{
    try
    {
        IBuffer fileBuffer = await FileIO.ReadBufferAsync(file);
        var inkDocument = Will3Codec.Decode(fileBuffer.ToArray());
...

 async openFile(buffer) {
   let inkModel = this.codec.decodeInkModel(buffer);
...

处理模型​

Ink Model​

输入上下文数据库​

环境​

InkInputProvider​

输入设备​

SensorContext​

SensorData 版本库​

Stroke 版本库​

Brush 版本库​

暂留​

编码​

解码​