New drawing library - drawing.js

Introduction

One of the fundamental concerns of cornerstoneTools is drawing markings on top of an image canvas.

The fundamental object used to achieve this is the HTML Canvas 2D Context (or context for short).

Problems

1. Required knowledge

In order to write a new tool which draws to the canvas, the developer must familiarise themselves with the concept of a context and learn the API and how to correctly use it.

Directly using the context API involves a large amount of boiler plate code to correctly initiate the context, save/restore the context state stack, and begin/end paths.

Example: Drawing a line between two points

    context.save();
    context.beginPath();
    context.strokeStyle = color;
    context.lineWidth = lineWidth;
    context.moveTo(start.x, start.y);
    context.lineTo(end.x, end.y);
    context.stroke();
    context.restore();

The above example demonstrates how to draw a line between two points with a given color and width. The developer needs to know about 6 different API methods and 2 API attributes in order to achieve a simple task.

2. Interleaving the what and the how

The verbose context API encourages a pattern of interleaving the calculation of what to display with the API calls saying how to display it. For example

    context.save();
    context.beginPath();

    const color = getColor();

    context.strokeStyle = color;

    const lineWidth = getLineWidth();

    context.lineWidth = lineWidth;

    const start = getStart();

    context.moveTo(start.x, start.y);
  
    const end = getEnd();
   
    context.lineTo(end.x, end.y);

    context.stroke();
    context.restore();

This makes it difficult to follow exactly what is being done on the canvas, as the single conceptual item (draw a line) has been interspersed with other lines of code.

3. Duplication

Working directly with the context API means that we have a large amount of duplication, as every time we want to draw a line (or any other common pattern), we need to write the same 8 lines of code.

Proposed Solution

In order to address these issues, I am proposing to introduce a module in util/drawing.js which will provide a simple API which matches the existing use cases of cornerstoneTools.

This module will be used internally and and also exposed as part of the public interface to allow developers to write custom tools more easily.

API

getNewContext(canvas)

Create a new context object and set the transform to the identity transform.

draw(context, fn)

This function manages the save/restore pattern for working in a new context state stack. The parameter fn is passed the context and can execute any API calls in a clean stack.

Example

draw(context, (context) => {
    drawLine(context, ...);
    drawCircle(context, ...);
});

path(context, options, fn)

This function manages the beginPath/stroke pattern for working with path objects.

Options

• color: The color (or any other valid strokeStyle of the path.
• lineWidth: The width of lines in the path. If null, no line width is set. If undefined then toolStyle.getToolWidth() is set.
• fillStyle: The fillStyle to fill the path with. If undefined then no filling is done.
• lineDash: The dash pattern to use on the lines

Example

path(context, { color }, (context) => {
    context.moveTo(start.x, start.y);
    context.lineTo(end.x, end.y);
});

setShadow(context, options)

Set the shadow properties of the context. Each shadow is set on the context object if defined, otherwise a default value is set.

Options

• shadow: Boolean value. If false, then shadow values are set.
• shadowColor: Default value: #000000
• shadowOffsetX: Default value: 1
• shadowOffsetY: Default value: 1

drawLine(context, element, start, end, options, coordSystem = 'pixel')

Draw a line between start and end.

• start, end: { x, y } in either pixel or canvas coordinates.
• options: Same as for path()
• coordSystem: Can be "pixel"(default) or "canvas". The coordinate system of the points passed in to the function. If "pixel" then cornerstone.pixelToCanvas is used to transform the points from pixel to canvas coordinates.

drawLines(context, element, lines, options, coordSystem = 'pixel')

Draw multiple lines.

• lines: [{ start: {x, y}, end: { x, y }] An array of start, end pairs. Each point is { x, y } in either pixel or canvas coordinates.
• options: Same as for path()
• coordSystem: Can be "pixel"(default) or "canvas". The coordinate system of the points passed in to the function. If "pixel" then cornerstone.pixelToCanvas is used to transform the points from pixel to canvas coordinates.

drawJoinedLines(context, element, start, points, options, coordSystem = 'pixel')

Draw a series of joined lines, starting at start and then going to each point in points.

• start: { x, y } in either pixel or canvas coordinates.
• points: [{ x, y }] An array of points in either pixel or canvas coordinates.
• options: Same as for path()
• coordSystem: Can be "pixel"(default) or "canvas". The coordinate system of the points passed in to the function. If "pixel" then cornerstone.pixelToCanvas is used to transform the points from pixel to canvas coordinates.

drawCircle(context, element, center, radius, options, coordSystem = 'pixel')

NOTE: This replaces the existing drawCircle() function

Draw a circle with given center and radius.

• center: { x, y } in either pixel or canvas coordinates.
• radius: The circles radius in canvas units.
• options: Same as for path()
• coordSystem: Can be "pixel"(default) or "canvas". The coordinate system of the points passed in to the function. If "pixel" then cornerstone.pixelToCanvas is used to transform the points from pixel to canvas coordinates.

drawEllipse(context, element, corner1, corner2, options, coordSystem = 'pixel')

NOTE: This replaces the existing drawEllipse() function

Draw an ellipse within the bounding box defined by corner1 and corner2.

• corner1, corner2: { x, y } in either pixel or canvas coordinates.
• options: Same as for path()
• coordSystem: Can be "pixel"(default) or "canvas". The coordinate system of the points passed in to the function. If "pixel" then cornerstone.pixelToCanvas is used to transform the points from pixel to canvas coordinates.

drawRect(context, element, corner1, corner2, options, coordSystem = 'pixel')

Draw a rectangle defined by corner1 and corner2.

• corner1, corner2: { x, y } in either pixel or canvas coordinates.
• options: Same as for path()
• coordSystem: Can be "pixel"(default) or "canvas". The coordinate system of the points passed in to the function. If "pixel" then cornerstone.pixelToCanvas is used to transform the points from pixel to canvas coordinates.

fillBox(context, boundingBox, fillStyle)

Draw a filled rectangle defined by boundingBox using the style defined by fillStyle

• boundingBox: { left, top, width, height } in canvas coordinates.
• fillStyle: The fillStyle to apply to the region.

fillTextLines(context, boundingBox, textLines, fillStyle, padding)

Draw multiple lines of text within a bounding box.

• boundingBox: { left, top } in canvas coordinates. Only the top-left corner is specified, as the text will take up as much space as it needs.
• textLines: [String] An array of strings of text.
• fillStyle: The fillStyle to apply to the text.
• padding: The amount of padding above/below each line in canvas units. Note this gives an inter-line spacing of 2 * padding.

Design Decisions

A number of design considerations have gone into the proposed API.

• Minimal: The API has the least number of functions that is practical. Each function has at least one possible use case within the existing code base.
• Consistent: Where possible, the available parameters and their meanings are consistent across all functions in the API.
• Sufficient: The proposed API covers over 99% of all existing use cases of the context API.

Proposed Deployment

Taking advantage of the new API will require significant changes to a large portion of the existing code base. Doing this work in a single PR is not feasible as it would require reviewing many interacting changes all at once, which is likely to let bugs slip through. As such, I propose that the changes go through in a series of PRs.

• 0/N: I will create a PR which contains all the prototype work which has lead to the development of the new API. This PR will contain a branch which has been completely refactored to use the API, but which almost definitely contains bugs and regressions. It can be used as a reference point for the general direction of the changes and to anticipate any potential problems which might arise. This PR will never get merged.
• 1/N: Add drawing.js with the complete API and remove the existing drawCircle.js and drawEllipse.js functions, which are superseded by new API functions. This PR will require at least a minor version number bump, as the public facing API will be changed.
• 2/N: Use getNewContext in all appropriate places.
• 3/N: " draw " " "
• 4/N: " path " " "
• .../N: One PR each for the remaining API functions
• .../N: Refactor existing code to remove extra code from inside path blocks.
• .../N: Refactor existing code to remove extra code from inside draw blocks.

Discussion

While I have presented what appears to be a definitive proposal I welcome a discussion about any and all aspects of this proposal. For a large scale change like this it is important that we get the design right, rather than get it done quickly.

I will submit PRs 0/N and 1/N to provide a concrete implementation to serve as a reference point, but I suggest that we do not merge 1/N until there is consensus on the design.

Finally I would like to thank all the developers who have made this amazing tool available for everyone to use. It has already proved invaluable to our development team and I hope that by contributing these changes we can give back to the rest of the community.

[timleslie]

So my main comment is to consider how this API would look if it wasn’t using Canvas rendering under the hood. Would it still make sense to pass around the context?

I think the proposed API is not going to be workable with SVG without a fundamental paradigm shift. In particular, the draw() and path() functions in the proposed API exist purely to manage the state within the context object of the canvas drawing API. I don’t think the same patterns will apply in an SVG API and I suspect there will be quirks of the SVG API itself which don’t lend themselves the current pattern.

While I agree that it would be good to include support for SVG drawing within cornerstoneTools, I think it should be kept out of scope for the current set of changes. Once the proposed drawing API is implemented and all the builtin tools are using it, I think we’ll be in a better position to consider what an SVG API should look like and we can think about consolidating the APIs where appropriate from there.

I don’t think we need to remove drawCircle just yet. We can just make the other tools import the new drawCircle from somewhere else (drawing folder), and mark the old one with @deprecated.

Fair enough. I have updated the 1/N PR and left these functions in. For now I have simply left a @deprecated comment. I’m happy to add a more specific mechanism for raising errors if that’s what you’d prefer.

The 'element: If undefined then the points are assumed to be in canvas coordinates. ’ will probably be confusing for many people. Might be better to make the coordinate system an option instead.

Yep, good call. I’ve updated the proposal and the code in the 1/N PR to include a coordSystem argument, which can be set to pixel or canvas so that we don’t have strange implicit behaviour based on the status of the element argument.

[diego0020]

I am not sure about the performance of SVG vs. Canvas.

In fact I remember D3 made the switch from SVG to Canvas and the performance improved significantly. https://bl.ocks.org/john-guerra/e80f02e59681ace122626c39407526a0 However they deal with thousands of lines or points. I guess in cornerstone the number of measurements will never be that large.