Skip to main content

墨水几何结构和渲染管线

我们现在引入了不同的rendering modes。 该指南对管道设置和渲染每一步坐了详细介绍。

视觉结果取决于刷子配置、管道配置以及平台渲染配置。

Raster Ink

本节逐步详细介绍了最佳的几何图形设置方法和栅格墨水的渲染管道。

笔刷配置

要设置光栅颗粒刷子,需要两种纹理:

形状和填充纹理

形状纹理和填充纹理:

shape参数指定颗粒的形状。 它接受图像,可以不透明像素定义颗粒形状。

fill参数指定形状的填充图案。 它接受图像,可从中采样图案。

两种纹理都可以在该参数中传递一系列图像。 这种情况下,该方法将它们解析为纹理映射。

...
fun pencil(context: Context): RasterBrush {
val opts = BitmapFactory.Options()
opts.inSampleSize = 1
opts.inScaled = false
// Texture for shape
val shapeTexture =
BitmapFactory.decodeResource(context.resources, R.drawable.essential_shape, opts)
// Texture for fill
val fillTexture =
BitmapFactory.decodeResource(context.resources, R.drawable.essential_fill_11, opts)

// Convert to PNG
val stream = ByteArrayOutputStream()
shapeTexture!!.compress(Bitmap.CompressFormat.PNG, 100, stream)
val shapeTextureByteArray = stream.toByteArray()

val stream2 = ByteArrayOutputStream()
fillTexture!!.compress(Bitmap.CompressFormat.PNG, 100, stream2)
val fillTextureByteArray = stream2.toByteArray()

// Create the raster brush
var brush = RasterBrush(
URIBuilder.getBrushURI("raster", "Pencil"), // name of the brush
0.15f, // spacing
0.15f, // scattering
RotationMode.RANDOM, // rotation mode
listOf(shapeTextureByteArray), // shape texture
listOf(), fillTextureByteArray, // fill texture
"", // fill texture URI
fillTexture.width.toFloat(), // width of texture
fillTexture.height.toFloat(), // height of texture
false, // randomized fill
BlendMode.MAX // mode of blending
)

shapeTexture.recycle()
fillTexture.recycle()

return brush
}

...

PathPoint 计算器

对于所有相关的刷子,需要配置PathPoint 计算器 ,它会影响传感器数据更改墨水笔划宽度、强度和倾斜度的方式。

首先,需要配置布局。 布局可根据输入提供者调整。 例如,对于触摸输入,不接收倾斜度或压力信息。

...

import com.wacom.ink.PathPointLayout

...

override fun getLayout(): PathPointLayout {
// Define different layouts for stylus and touch input
if (isStylus) {
return PathPointLayout(
PathPoint.Property.X,
PathPoint.Property.Y,
PathPoint.Property.SIZE,
PathPoint.Property.ALPHA,
PathPoint.Property.ROTATION,
PathPoint.Property.OFFSET_X
)
} else {
return PathPointLayout(
PathPoint.Property.X,
PathPoint.Property.Y,
PathPoint.Property.SIZE,
PathPoint.Property.ALPHA
)
}
}
...

...

companion object {
val uri = URIBuilder.getToolURI("raster", "pencil")
// Minimum size of the pencil tip
val MIN_PENCIL_SIZE = 4f
// Maximum size of the pencil tip
val MAX_PENCIL_SIZE = 25f
// Minimum alpha values for the particles
val MIN_ALPHA = 0.1f
// Maximum alpha values for the particles
val MAX_ALPHA = 0.7f
// Unit for speed is px/second.
// NOTE: This needs to be optimized for different Pixel densities of different devices
val MAX_SPEED = 15000f
}

...

override val touchCalculator: Calculator = { previous, current, next ->
// Use the following to compute size based on speed:
var size = current.computeValueBasedOnSpeed(
previous,
next,
minValue = MIN_PENCIL_SIZE,
maxValue = MAX_PENCIL_SIZE,
minSpeed = 80f,
maxSpeed = MAX_SPEED,
// reverse behaviour
remap = { 1f - it}
)!!

var alpha = current.computeValueBasedOnSpeed(
previous,
next,
minValue = MIN_ALPHA,
maxValue = MAX_ALPHA,
minSpeed = 80f,
maxSpeed = MAX_SPEED,
// reverse behaviour
remap = { 1f - it}
)

if (alpha == null) {
alpha = previousAlpha
} else {
previousAlpha = alpha
}

PathPoint(current.x, current.y, size = size, alpha = alpha)
}

触控笔输入配置可以更方便地配置现实世界书写工具的自然行为。 可在此通过压力传感器数据增加工具的宽度或更改颜色的强度。 另外,如果触控笔倾斜,将自然而然地更改接触面积区域。 如果想要实现笔的影线,水彩效果或墨水笔效果,需要在此设置。

**注意:**输入传感器质量各不相同,因此,需要通过不同的触控笔输入测试是否达到所需的行为效果。

...

var previousAlpha = 0.2f

...
override val stylusCalculator: Calculator = { previous, current, next ->
// calculate the offset of the pencil tip due to tilted position
val cosAltitudeAngle = cos(current.altitudeAngle!!)
val sinAzimuthAngle = sin(current.azimuthAngle!!)
val cosAzimuthAngle = cos(current.azimuthAngle!!)
val x = sinAzimuthAngle * cosAltitudeAngle
val y = cosAltitudeAngle * cosAzimuthAngle
val offsetY = 5f * -x
val offsetX = 5f * -y
// compute the rotation
val rotation = current.computeNearestAzimuthAngle(previous)
// now, based on the tilt of the pencil the size of the brush size is increasing, as the
// pencil tip is covering a larger area
val size = max(MIN_PENCIL_SIZE,
min(MAX_PENCIL_SIZE, MIN_PENCIL_SIZE + 20f * abs(current.altitudeAngle!!)))
// Change the intensity of alpha value by pressure of speed, if available else use speed
var alpha = if (current.force == -1f) {
current.computeValueBasedOnSpeed(
previous,
next,
minValue = MIN_ALPHA,
maxValue = MAX_ALPHA,
minSpeed = 0f,
maxSpeed = 3500f,
// reverse behaviour
remap = { 1.0f - it}
)
} else {
current.computeValueBasedOnPressure(
minValue = MIN_ALPHA,
maxValue = MAX_ALPHA,
minPressure = 0.0f,
maxPressure = 1.0f,
remap = { v: Float -> v.toDouble().pow(1).toFloat() }
)
}

if (alpha == null) {
alpha = previousAlpha
} else {
previousAlpha = alpha
}
PathPoint(
current.x, current.y,
alpha = alpha, size = size, rotation = rotation,
offsetX = offsetX, offsetY = offsetY
)
}
...

管线

RasterInkBuilder定义栅格墨水管道:

import com.wacom.ink.*
import com.wacom.ink.format.rendering.RasterBrush
import com.wacom.ink.pipeline.PathProducer
import com.wacom.ink.pipeline.SmoothingFilter
import com.wacom.ink.pipeline.SplineInterpolator
import com.wacom.ink.pipeline.SplineProducer
import com.wacom.ink.pipeline.base.ProcessorResult

...
class RasterInkBuilder() {

// The pipeline components:
lateinit var pathProducer: PathProducer
lateinit var smoother: SmoothingFilter
lateinit var splineProducer: SplineProducer
lateinit var splineInterpolator: SplineInterpolator
lateinit var pathSegment: PathSegment
lateinit var tool: RasterTool

...

fun updatePipeline(newTool: Tool) {
if (newTool !is RasterTool) {
throw IllegalArgumentException("Invalid tool")
} else {
tool = newTool
var layout = tool.getLayout()
// Path producer needs to know the layout
pathProducer = PathProducer(layout, tool.getCalculator())
smoother = SmoothingFilter(layout.size) // Dimension
splineProducer = SplineProducer(layout) // Layout
splineInterpolator = SplineInterpolator(
spacing = tool.brush.spacing, // Spacing between two successive sample.
splitCount = 1, // Determines the number of iterations for the
// discretization.
calculateDerivatives = true, // Calculate derivatives
interpolateByLength = true // Interpolate by length
)
pathSegment = PathSegment()

splineProducer.keepAllData = true
}
}
}
...

光栅化视图

接下来,设置光栅视图:

import com.wacom.ink.egl.EGLRenderingContext
import com.wacom.ink.format.tree.data.Stroke
import com.wacom.ink.format.tree.nodes.StrokeNode
import com.wacom.ink.rasterization.InkCanvas
import com.wacom.ink.rasterization.Layer
import com.wacom.ink.rasterization.StrokeRenderer
import com.wacom.ink.rendering.BlendMode
...

/**
* This is a surface for drawing raster inking.
* Extends from SurfaceView
*/
class RasterView @JvmOverloads constructor(
context: Context,
attrs: AttributeSet? = null,
defStyleAttr: Int = 0
) : SurfaceView(context, attrs, defStyleAttr) {

...

private var inkCanvas: InkCanvas? = null
private lateinit var viewLayer: Layer
private lateinit var strokesLayer: Layer
private lateinit var currentFrameLayer: Layer
private lateinit var strokeRenderer: StrokeRenderer

private var rasterInkBuilder = RasterInkBuilder() // The ink builder
var rasterTool: RasterTool = PencilTool(context) // The default tool

...

init {
rasterInkBuilder.updatePipeline(PencilTool(context))
setZOrderOnTop(true);
holder.setFormat(PixelFormat.TRANSPARENT);

// the drawing is going to be performer on background
// on a surface, so we initialize the surface
holder.addCallback(object : SurfaceHolder.Callback {
override fun surfaceChanged(holder: SurfaceHolder?, format: Int, w: Int, h: Int) {
// here, once the surface is crated, we are going to initialize ink canvas

// firs we check that there is no other inkCanvas, in case there is we dispose it
if (inkCanvas?.isDisposed == false) {
releaseResources();
}

inkCanvas =
InkCanvas(holder, EGLRenderingContext.EGLConfiguration(8, 8, 8, 8, 8, 8))
viewLayer = inkCanvas!!.createViewLayer(w, h)
strokesLayer = inkCanvas!!.createLayer(w, h)
currentFrameLayer = inkCanvas!!.createLayer(w, h)

inkCanvas!!.clearLayer(currentFrameLayer)

strokeRenderer =
StrokeRenderer(inkCanvas, PencilTool(context).brush.toParticleBrush(), w, h)

drawStrokes(strokeNodeList)
renderView()

if (listener != null) {
listener.onSurfaceCreated()
}
}

override fun surfaceDestroyed(holder: SurfaceHolder?) {
releaseResources();
}

override fun surfaceCreated(p0: SurfaceHolder?) {
//we don't have to do anything here
}
})
}
}
...

输入处理

操作系统的输入事件使用下列方式进行处理:

// This function is going to be call when we touch the surface
fun surfaceTouch(event: MotionEvent) {
// Check the input method
if (event.resolveToolType() == InkInputType.PEN) {
if ((newTool) || (!isStylus)) {
newTool = false
isStylus = true
rasterInkBuilder.updateInputMethod(isStylus)
}
} else {
if ((newTool) || (isStylus)) {
newTool = false
isStylus = false
rasterInkBuilder.updateInputMethod(isStylus)
}
}

for (i in 0 until event.historySize) {
val pointerData = event.historicalToPointerData(i)
rasterInkBuilder.add(pointerData.phase, pointerData, null)
}

val pointerData = event.toPointerData()
rasterInkBuilder.add(pointerData.phase, pointerData, null)

val (added, predicted) = rasterInkBuilder.build()

if (pointerData.phase == Phase.BEGIN) {
// initialize the sensor data each time a new stroke begin
val pair = inkEnvironmentModel.createSensorData(event)
sensorData = pair.first
channelList = pair.second
}

for (channel in channelList) {
when (channel.type) {
InkSensorType.X -> sensorData.add(channel, pointerData.x)
InkSensorType.Y -> sensorData.add(channel, pointerData.y)
InkSensorType.TIMESTAMP -> sensorData.addTimestamp(
channel,
pointerData.timestamp
)
InkSensorType.PRESSURE -> sensorData.add(channel, pointerData.force!!)
InkSensorType.ALTITUDE -> sensorData.add(channel, pointerData.altitudeAngle!!)
InkSensorType.AZIMUTH -> sensorData.add(channel, pointerData.azimuthAngle!!)
}
}

if ((pointerData.phase == Phase.END) && (rasterInkBuilder.splineProducer.allData != null)) {
addStroke()
sensorDataList.add(sensorData)
}


if (added != null) {
drawStroke(event, added, predicted)
}

renderView()
}

/// <summary>
/// Initiates capture of an ink stroke
/// </summary>
private void OnPointerPressed(object sender, PointerRoutedEventArgs args)
{
// If currently there is an unfinished stroke - do not interrupt it
if (!m_pointerManager.OnPressed(args))
return;

m_swapChainPanel.CapturePointer(args.Pointer);

StrokeHandler.OnPressed(m_swapChainPanel, args);
}

/// <summary>
/// Updates current ink stroke with new pointer input
/// </summary>
private void OnPointerMoved(object sender, PointerRoutedEventArgs args)
{
// Ignore events from other pointers
if (!m_pointerManager.OnMoved(args))
return;

StrokeHandler.OnMoved(m_swapChainPanel, args);
}

/// <summary>
/// Completes capture of an ink stroke
/// </summary>
private void OnPointerReleased(object sender, PointerRoutedEventArgs args)
{
// Ignore events from other pointers
if (!m_pointerManager.OnReleased(args))
return;

m_swapChainPanel.ReleasePointerCapture(args.Pointer);

StrokeHandler.OnReleased(m_swapChainPanel, args);

if (!StrokeHandler.IsSelecting)
{
RenderNewStrokeSegment();

BlendCurrentStrokesLayerToAllStrokesLayer();

RenderBackbuffer();
PresentGraphics();

var ptrDevice = args.GetCurrentPoint(m_swapChainPanel).PointerDevice;
StrokeHandler.StoreCurrentStroke(ptrDevice.PointerDeviceType);
}
}

// StrokeHandler for raster strokes
public class RasterStrokeHandler : StrokeHandler
{
...

public override void OnPressed(UIElement uiElement, PointerRoutedEventArgs args)
{
m_startRandomSeed = (uint)mRand.Next();
base.OnPressed(uiElement, args);
ActiveTool.OnPressed(uiElement, args);
}

public override void OnMoved(UIElement uiElement, PointerRoutedEventArgs args)
{
ActiveTool.OnMoved(uiElement, args);
}

public override void OnReleased(UIElement uiElement, PointerRoutedEventArgs args)
{
ActiveTool.OnReleased(uiElement, args);
}

...
}
class InkCanvas extends InkController {

...

begin(sensorPoint) {
if (this.forward) return this.inkCanvasRaster.begin(sensorPoint);

this.reset(sensorPoint);

this.builder.add(sensorPoint);
this.builder.build();
}

move(sensorPoint) {
if (this.forward) return this.inkCanvasRaster.move(sensorPoint);

if (this.requested) {
this.builder.ignore(sensorPoint);
return;
}

this.builder.add(sensorPoint);

if (!this.requested) {
this.requested = true;

this.builder.build();

requestAnimationFrame(() => (this.requested = false));
}
}

end(sensorPoint) {
if (this.forward) return this.inkCanvasRaster.end(sensorPoint);

this.builder.add(sensorPoint);
this.builder.build();
}

...

Finally, draw the stroke:

fun drawStrokes(strokeList: MutableList<Pair<StrokeNode, RasterBrush>>) {
for (stroke in strokeList) {
drawStroke(stroke.first, stroke.second)
}
}

...

// Draw stroke
private fun drawStroke(
event: MotionEvent,
added: InterpolatedSpline,
predicted: InterpolatedSpline?
) {
strokeRenderer.drawPoints(added, defaults, event.action == MotionEvent.ACTION_UP)

if (predicted != null) {
strokeRenderer.drawPrelimPoints(predicted, defaults)
}

if (event.action != MotionEvent.ACTION_UP) {
inkCanvas!!.setTarget(currentFrameLayer, strokeRenderer.strokeUpdatedArea)
inkCanvas!!.clearColor()
inkCanvas!!.drawLayer(strokesLayer, BlendMode.SOURCE_OVER)
strokeRenderer.blendStrokeUpdatedArea(currentFrameLayer, rasterTool.getBlendMode())
} else {
strokeRenderer.blendStroke(strokesLayer, rasterTool.getBlendMode())
inkCanvas!!.setTarget(currentFrameLayer)
inkCanvas!!.clearColor()
inkCanvas!!.drawLayer(strokesLayer, BlendMode.SOURCE_OVER)
}
}
...

Vector Ink

本节将逐步详细介绍最佳的几何图形设置方法和矢量墨水的渲染管道。

笔刷配置

矢量墨水刷子配置由任意凸多边形定义。

Polygon

import com.wacom.ink.format.enums.RotationMode
import com.wacom.ink.format.rendering.RasterBrush
import com.wacom.ink.rendering.BlendMode
import com.wacom.ink.rendering.BrushPrototype
import com.wacom.ink.rendering.VectorBrush
...
class BrushPalette {
...

fun circle(): VectorBrush {
return VectorBrush(
"circle", // inSampleSize
arrayOf(
BrushPrototype(FloatArrayList(ShapeFactory.createCircle(4)), 0f),
BrushPrototype(FloatArrayList(ShapeFactory.createCircle(8)), 2f),
BrushPrototype(FloatArrayList(ShapeFactory.createCircle(16)), 6f),
BrushPrototype(FloatArrayList(ShapeFactory.createCircle(32)), 18f)
)
) // Brush prototypes for different level of details
}

...

class ShapeFactory {

companion object {

/**
* Creates polygon with circle shape
*
* @param {int} [n=20] number of polygon segments
* @param {int} [r=1] radius (0 < r <= 1)
* @param {Point} [c={x: 0, y: 0}] center
* @return {Polygon} circle shape points
*/
@JvmStatic
fun createCircle(n: Int = 20, r: Float = 1f,
c: PointF = PointF(0f, 0f)): FloatArray {
return createEllipse(n, r, r, c);
}

...

}
}

}

PathPoint 计算器

PathPoint计算器的定义方法与光栅工具的类似。 同样,我们需要定义PathPoints的布局:

override fun getLayout(): PathPointLayout {
if (isStylus) {
/**
* The currently layout will use:
* - Coordinate values (X and Y)
* - Size - the size of the brush at any point of the stroke
* - For tilt effect the rotation, scale y and offset y
*/
return PathPointLayout(
PathPoint.Property.X,
PathPoint.Property.Y,
PathPoint.Property.SIZE,
PathPoint.Property.ROTATION,
PathPoint.Property.SCALE_Y,
PathPoint.Property.OFFSET_Y
)
} else {
/**
* The currently layout will use:
* - Coordinate values (X and Y)
* - Size - the size of the brush at any point of the stroke
*/
return PathPointLayout(
PathPoint.Property.X,
PathPoint.Property.Y,
PathPoint.Property.SIZE
)
}
}

管线

VectorInkBuilder是一个使用管道处理路径构建的类。 它可以处理整个管道的输入,并通过输入产生多边形。 之后,多边形被转换为天然结构。

import com.wacom.ink.PathSegment
import com.wacom.ink.Phase
import com.wacom.ink.PointerData
import com.wacom.ink.Spline
import com.wacom.ink.pipeline.*
import com.wacom.ink.pipeline.base.ProcessorResult
...
class VectorInkBuilder() {

// The pipeline components:
internal lateinit var pathProducer: PathProducer
internal lateinit var smoother: SmoothingFilter
internal lateinit var splineProducer: SplineProducer
internal lateinit var splineInterpolator: SplineInterpolator
internal lateinit var brushApplier: BrushApplier
internal lateinit var convexHullChainProducer: ConvexHullChainProducer
internal lateinit var polygonMerger: PolygonMerger
internal lateinit var polygonSimplifier: PolygonSimplifier
internal lateinit var bezierPathBuilder: PolygonToBezierPathProducer
internal lateinit var pathSegment: PathSegment

lateinit var tool: VectorTool

fun updatePipeline(newTool: Tool) {
if (newTool !is VectorTool) {
throw IllegalArgumentException("Invalid tool")
} else {
tool = newTool
var layout = tool.getLayout()
pathProducer = PathProducer(layout, tool.getCalculator())
smoother = SmoothingFilter(
layout.count(), // Dimension
30) // Moving average window size
splineProducer = SplineProducer(layout) // Layout
splineInterpolator = SplineInterpolator(
tool.brush.spacing, // Spacing between two successive sample.
6, // Determines the number of iterations for the discretization.
false, // Calculate derivatives
false) // Interpolate by length
brushApplier = BrushApplier(
tool.brush, // Vector brush
true) // Keep all data
convexHullChainProducer = ConvexHullChainProducer()
polygonMerger = PolygonMerger()
polygonSimplifier = PolygonSimplifier(0.1f) // Epsilon
bezierPathBuilder = PolygonToBezierPathProducer()
pathSegment = PathSegment()
splineProducer.keepAllData = true
}
}
}
...

矢量化视图

接下来,需要设置矢量视图。 该类是一个用于绘制矢量墨水的简单视图。 矢量墨水是一条路径,它通过扩展平台渲染方法并绘制路径来绘制。

import com.wacom.ink.PathPoint
import com.wacom.ink.Phase
import com.wacom.ink.PointerData
import com.wacom.ink.Spline
import com.wacom.ink.format.InkSensorType
import com.wacom.ink.format.enums.InkInputType
import com.wacom.ink.manipulation.ManipulationMode
import com.wacom.ink.manipulation.SpatialModel
import com.wacom.ink.manipulation.callbacks.ErasingCallback
import com.wacom.ink.manipulation.callbacks.SelectingCallback
import com.wacom.ink.model.InkStroke
import com.wacom.ink.model.StrokeAttributes
...

class VectorView @JvmOverloads constructor(
context: Context,
attrs: AttributeSet? = null,
defStyleAttr: Int = 0
) : View(context, attrs, defStyleAttr) {

// Relevant for serialization
var inkSensorTypes = listOf(InkSensorType.X, InkSensorType.Y, InkSensorType.TIMESTAMP)
lateinit var inkEnvironmentModel: InkEnvironmentModel

var drawingTool: VectorTool = PenTool() //default tool
var inkBuilder = VectorInkBuilder()

// This will contain the current drawing path before it is finished
private var currentPath = Path()
var paint: Paint = Paint()

// A list of existing strokes
private var strokes = mutableMapOf<String, WillStroke>()

...

private fun addStroke(event: MotionEvent) {
val brush = inkBuilder.tool.brush
val stroke = WillStroke(inkBuilder.splineProducer.allData!!.copy(), brush)
stroke.sensorData = inkEnvironmentModel.createSensorData(event, inkSensorTypes)

stroke.strokeAttributes = object : StrokeAttributes {
override var size: Float = 10f
override var rotation: Float = 0.0f

override var scaleX: Float = 1.0f
override var scaleY: Float = 1.0f
override var scaleZ: Float = 1.0f

override var offsetX: Float = 0.0f
override var offsetY: Float = 0.0f
override var offsetZ: Float = 0.0f

override var red: Float = Color.red(paint.color) / 255f
override var green: Float = Color.green(paint.color) / 255f
override var blue: Float = Color.blue(paint.color) / 255f
override var alpha: Float = paint.alpha.toFloat() / 255
}

spatialModel.add(stroke)
stroke.path = currentPath
strokes[stroke.id] = stroke
}
}

输入处理

操作系统的输入事件使用与光栅输入相同的方式进行处理 (please see Raster Input Handling)

最后,绘制笔划:

/**
* In order to make the strokes look nice and smooth, it is recommended to enable:
* - Antialiasing
* - Dither
*/
val paint = Paint().also {
it.isAntiAlias = true
it.isDither = true
}
...
override fun onDraw(canvas: Canvas) {
super.onDraw(canvas)

selectionBox?.drawSelectionBox(canvas)

for ((id, stroke) in strokes) {

if (id in selectedStrokes) {
canvas.drawPath(stroke.path, paintSelected)
} else {
// Selection color
val color = Color.argb(
(stroke.strokeAttributes.alpha * 255).toInt(),
(stroke.strokeAttributes.red * 255).toInt(),
(stroke.strokeAttributes.green * 255).toInt(),
(stroke.strokeAttributes.blue * 255).toInt()
)
paint.color = color
canvas.drawPath(stroke.path, paint)
}
}

// draw the current path,
// because we have not finished the stroke we has not saved it yet
if (drawingTool.uri() == EraserVectorTool.uri) {
paint.color = Color.WHITE
} else if (drawingTool.uri() == EraserWholeStrokeTool.uri) {
paint.color = Color.RED
} else {
paint.color = currentColor
}
canvas.drawPath(currentPath, paint)
}