SketchIron: Workflow for Vector-based
Sketching
Thesis Master Information Science
Track Human Centered Multimedia
University of Amsterdam
Faculty of Science
Author: Timur Carpeev
Student number: 10652418
Supervisor: dhr. prof. dr. Marcel Worring
SketchIron: Workflow for Vector-based Sketching
Timur Carpeev
Student ID 10652418
Faculty of Science, University of Amsterdam timur.carpeev@gmail.com
ABSTRACT
Conventional vector drawing editors are designed around mathematical models of manipulating vector shapes which are not suitable for quick sketching. Some systems facilitate free hand vector sketching tools, however there are gaps in the overall experience of the sketching workflow. In this paper, we present a system for sketching with vector data that allows users to apply free hand sketching techniques to manipulate existing strokes and apply beautification transformations as they draw. Our system improves several aspects of the sketching workflow compared to mainstream tools, as confirmed by the feedback collected from a group of artists.
Categories and Subject Descriptors
H.5.2 Graphical User Interfaces, Input Devices, and Strategies; D.2.2User Interfaces; I.3.6 Interaction Techniques
General Terms
Algorithms, Design, Human Factors.
Keywords
Beautification assistant, vector drawing, sketching, interaction design, vector editor, stroke based interfaces.
1. INTRODUCTION
For typical artists, the process of sketching is a quick way to visualize ideas without worrying about the details. By loosely drawing shapes with imprecise strokes, artists build a foundation that can later be detailed further or used to evaluate ideas. Such early stage drawings are well suited for free hand gestures and are often done with pen and paper or digital tools that simulate the analogue medium. In the field of digital illustration, drawing with existing mainstream vector tools requires precision and interaction with mathematical mental models which align quite poorly with the sketching process. Because of the complexity and effort required to use vector tools, illustrators typically use them only in the advanced stages of their process by drawing on top of the analogue sketches. In some workflow scenarios, however, sketching directly with vector tools is convenient, for instance when working with animations, or modifying existing vector art work. Existing vector tools are not specifically designed to support quick and rough sketching.
Multiple systems and algorithms were proposed to improve the experience of digital sketching in various graphic domains (Jorge, Joaquim, and Faramarz Samavati 2010) (Olsenet et al., 2008). Algorithms of such systems are described in great detail, however they lack insights into the extent to which such systems fit within the existing workflow of their users. Moreover, these systems are,
for the most part, evaluated only by their authors. This gap is mentioned in an observational study on sketching (Grimm, C. 2011), where the authors conclude that computer-aided sketching received a lot of attention, but the way artists interact with the medium tends to be explored much less. They argue the following: ’there has been a focus on how to manipulate
individual curves and strokes, but not on understanding and exploiting the semantics and stages of the drawing process’. In
addition to the lack of user insights of such systems, their code is not publicly available, which makes it difficult to conduct further studies on their usefulness.
The gap between research in computer-aided sketching and exploration of artists’ workflow motivates us to create SketchIron, a system for vector-based sketching. SketchIron comprises two drawing modes: a standard free hand drawing mode and a beautification mode that uses state-of-the-art techniques such as path smoothing and line grouping. The system was designed to better accommodate the artists’ sketching workflow, through reducing the need of using administrative controls.
Whereas existing vector drawing tools already incorporate a wide variety of features which allow the user to gain full control over the drawing output, these are arguably too distracting to fit into the natural sketching process of artists. In order to closer resemble the analogue drawing process, we believe that reducing the need for administrative controls can have a positive impact on the sketching scenario, even if it comes at the cost of reduced functionality of the drawing tool. We thus make the deliberate choice of minimizing the need for administrative controls in SketchIron.
By creating and testing SketchIron, our goal is to explore how vector editing tools can be designed to better fit the artist’s workflow and how this affects the user acceptance of such tools for the process of quick and rough sketching. It is worth noting that although we incorporate state-of-the-art vector drawing techniques into SketchIron, our goal is not to test or validate the efficiency of these techniques, but rather explore how they can support the digital drawing workflow.
For demonstrative purposes, our system is developed to run in a web browser, in order to allow setup-free and cross platform access. Web based standards are ubiquitous and give interested parties the convenience to extend and modify our code without the hassle of installing or compiling our project, since it can be done directly in the web browser. Our code is developed on top of the open source library Paper.js which is well documented and gives multiple primitives for building interactive 2D graphics. We believe that open and simple access to the system would help our readers gain a better understanding of the presented material and hopefully inspire new ideas for further research in this area.
The rest of this paper is structured as follows: Section 2 outlines the related work in the field of digital sketching. Section 3 describes our system design and features, followed by the implementation details in section 4. In section 5, we provide an evaluation of the system performed by professional and amateur illustrators, and supply an observational video with the evaluation process. Section 6 discusses the limitations of this research and potential future work, and section 7 presents the conclusions.
2. RELATED WORK
There are a wide range of ideas explored in the field of digital sketching. While each of them targets specific domains of applications, such as diagram composing or 3D modeling, the proposed techniques are typically useful in a wide range of graphic authoring domains. Below, we provide an overview of influential techniques and explain their relevance to our domain. We categorize existing papers according to the underlying vector drawing processing techniques: curve smoothing, beautification, stroke clustering and spline editing interactions.
Curve smoothing. Curve fitting algorithms approximate
imperfectly drawn lines to smoother and aesthetically pleasing parametric curves. This is an important aspect of line processing, since clear and smooth lines are common in the aesthetics of vector graphics (McCrae et al., 2011) (Frisken, Sarah F. 2008). A method was proposed by Brandt et al. (2015) to fit a spline through given points by minimizing the amount of segments of the curve. The algorithm makes a tradeoff between the minimum amount of segments of the curve and its deviation from the original form. Baran (2010) describes an algorithm for fitting clothoid splines to the given input, which results in seamless curved lines while also minimizing the deviation from the original input. Thiel et al. (2011) explores real-time stroke smoothing techniques that react to the speed of the user’s input. These techniques result in various strength of smoothing across the stroke, in order to give users the ability to keep the precision when needed by slowing down their hand movements. Such an approach gives the user control over curve smoothing in real-time, which aligns well with the ergonomics of sketching.
Beautification systems apply geometric constrains to a given sets
of strokes (Arvo & Novins, 2000). Such processing can be helpful for users aiming to achieve geometric looking representations, such as graphs, industrial design schemas, logo and icon design, etc (Zeleznik et al., 2008) (Wang 2005) (Cheema et al., 2012). Paulson & Hammond (2008) proposed a system that recognizes and quantizes up to 8 primitive shapes and adds constrains for connecting these shapes. Igarashi (1997) and Fišer (2015) propose a beautification assistant that supports various geometric rules such as endpoint snapping, line parallelism, perpendicularity, line length equality, curve offset, symmetry etc. This assistant allows to create polished-looking illustrations from imprecise and loosely drawn sketches. For schematic and geometric oriented work, such a system can produce desired outputs, however automatically applying such constraints to the sketches can be ambiguous and can produce unintended results (Pavlidis 1985) since it creates assumption about user intentions. Murugappan (2009) and Igarashi (1997) address the issue of ambiguous hand strokes by giving snapshots of multiple possible beautification outcomes from which users can pick according to their intention.
Stroke clustering is a technique which groups multiple strokes
into uniform lines, in order to produce cleaner and simplified results (Wang 2014). Clustering strokes can be useful for
interactions where users want to construct long curves from multiple short strokes, or it can be used to process sketches into line artwork (Pusch, 2007). Barla (2005) suggest a system of grouping lines based on a scale factor, which allows to the reduce line density of the artwork, making it possible to vary the level of details at scale to increase the visual clarity of the work. Orbay (2011) suggest a supervised stroke clustering algorithm which can learn from labeled example sketches with relationships between strokes. Such an approach requires supervised training, but can be adapted to various drawing styles. Most of the mentioned systems in this section can do order-independent grouping of collection of strokes. This means that it is possible to process the final composition after it is being complete. Some systems, however, join strokes only in a chronological order. In the context of this research, such techniques help users progressively build seamless long strokes with shorter ones. Progressively building strokes requires less precision from individual strokes compared to continuous hand movements.
Spline editing interactions. Baudel (1994) proposes a sketch-like
interface for working with spline curves. Instead of using mathematical representations of curves, it provides users with the ability to modify splines by overdrawing them similarly to how it is done on paper. The system describes basic stroke operations such as Create, Edit, Delete (part of the spline), and Join (make paths closed) by means of stroke patches. Such interactions help users stay in their flow of sketching without switching to different commands and modes. Henzen (2005) covers similar ideas of ovesketching lines to mimic paper and pen-like interaction.
3. OUR APPROACH
The features of our system are, to a large extent, based on the techniques from earlier work in this area, as covered in section 2. Namely, we use selected techniques for curve smoothing, beautification, stroke clustering and spline editing interactions in order to simulate the analogue drawing interactions as closely as possible.
• Curve smoothing forms the basis of SketchIron’s algorithm, which uses the drawing speed as an indicator of intended precision and thus of the required level of smoothing. • Beautification techniques are partially used by SketchIron,
such as endpoint snapping. The applicability of beautification features for our system is, however, limited, namely because they mostly focus on geometric shapes, whereas SketchIron is aimed at supporting free-hand drawing styles.
• Stroke clustering is applied in our system in a chronological order, since it will allow immediate feedback, thus making the results more explicit and measurable.
• Spline editing interactions stand at the core of SketchIron’s
interaction patterns, because they are aimed at recreating the analogue-like drawing process.
In order to make these techniques applicable for our particular goal and scope, we adapted or modified them as described in the rest of this section.
Our goal is to design a beautification system that can be used in the context of a large variety of drawing styles. For instance, some styles can be based on geometric and highly structured contours and thus benefit from basic shape quantization, while others are more organic or curvy and use hatching techniques. Based on this requirement we provide two drawing modes:
• Standard mode without any transformations except for line smoothing, which allows for a high degree of freedom. • Beautification mode for easy creation of continuous lines and
corrections.
The pipeline (Figure 1) of the application is composed of the following steps:
1. User input. Raw data of the user input is being recorded and
drawn on the screen. No transformations to the input is being applied at the point of drawing. During the the input, the timestamps are being collected for later calculation of speed of drawing.
2. Target identification. If the user holds the beautification mode
button, the closest (below a certain threshold) path is being identified from the location of the first point of the input. If the target path is detected, then it is being highlighted to provide feedback to the user.
3. Input smoothing. As soon as the user releases the pen from the
tablet, the path is being simplified according to the speed it was drawn with.
4. Gesture classification. Gestures classification distinguishes
between over-strokes, path extensions and a combination of both. Overall there are 4 classifiers that are later useful for applying appropriate transformation.
5. Stroke transformation. Based on the gesture classification, the
appropriate mark grouping algorithm is applied to the original input. From the perspective of the user, the transformation is happening instantly from the the moment of the release of the pen.
Figure 1. Input processing
We believe the user can benefit from constant switching between the two modes, since this would allow him to maintain direct control over the drawing process and the produced output. We strive, however, to reach a balance between user control and sketching ergonomics, thus we believe that using only one key to switch between the modes is optimal. The beautification mode in our system is triggered by holding a key while drawing, and the key can be assigned by the user. Typically, digital pens have buttons that are easily accessible (see Figure 2) while drawing and since there is only one key necessary for triggering beautification transformations, the user can use pen buttons to conveniently trigger the beautification mode. Requiring the user to trigger a key to inform about his intentions gives control over the output, however it comes at the expense of extra effort.
Figure 2. Digital pencil.
Below we describe in details the features of our system and their interaction flow.
Figure 3. Oversketching transformation examples. Oversketching strokes. While sketching, artists often overdraw
their lines, usually with a darker color, to provide corrections or to emphasize curvatures. Overdrawing strokes results in undesired noise and clutter. We thus decided to make it possible for artists to correct their strokes by drawing over a correcting stroke. To our knowledge, existing systems provide such possibility by completely replacing part of the stroke with a correcting stroke. In the analog medium, the correcting strokes vary in their strength depending on the desired emphasis. Therefore, we provide users with an option to control the strength of their correcting strokes. The maximum strength of the correction stroke is 1, which would result in the complete replacement of existing strokes with the correcting stroke. A strength of 0,5 (see Figure 3), for example, would result in the average between the existing and the correcting stroke. In addition to the strength of the transformation, correcting strokes get smoother based on the speed with which they are created. Thus the user has two degrees of control over correcting strokes - its strength and smoothness, the later can be achieved by using only hand gestures. It would be beneficial to use pressure sensitive information to dynamically manipulate the strength of the correcting strokes, but unfortunately due to our technical limitations this parameter can only be controlled manually.
The first step of applying overstrokes is to identify which part of the original stroke would be corrected. This is done by identifying edge points on either the original stroke or the modifying stroke, and then finding the corresponding closest points. The segment of the original stroke contained between these points will form the reflecting part, while the segment between the points on the modifying stroke will represent the correcting part (see Figure 4).
Figure 4. Correcting part identification.
The next step is to interpolate the correcting part with the reflected part. Each stroke is composed of segment points (see Figure 9). The interpolation algorithm can be visualized as shown in Figure 5.
Figure 5. Interpolation.
In next step we have to replace the fragment of the original stroke with the interpolated one. In order to blend in the interpolated stroke smoothly, we smooth parts of the original stroke at the position of the points where the interpolated stroke connects with original strokes. We define a variable for smoothing radius which we control in settings of the application. However, we avoid smoothing the regions that are laying on the edges of the stroke since edges can keep distinct information about the shape (see Figure 6).
Figure 6. Blending strokes.
Target detection. Before applying a stroke correction, the user
needs to select the targeted stroke. In some systems users need to explicitly select the line using a selection tool. In our system, when the user switches to the beautification mode, we detect the target strokes automatically based on the proximity of the cursor. When the user is moving a cursor, we highlight the closest strokes to its position to indicate that the line can be modified (see Figure 7). This feature helps users seamlessly switch between correction and drawing mode.
Figure 7. Target detection feedback.
Mark grouping. We define marks as a single continuous contact
movement of the digital input device. In the beautification drawing mode, single marks that are close to each other are grouped into strokes. All marks that begin within a customizable proximity threshold to other strokes are being blended in. This allows users to continuously extend the existing strokes with multiple short marks. We define several scenarios of grouping marks. First, when two marks are intersecting each other, from the intersection point the shortest segments of each mark are being trimmed (see Figure 8, first column). In the second scenario (see Figure 8, second column) when marks are not intersecting, they are being extended first, and if an intersection point is found they are being processed as in the first scenario. The extension is acting in the role of grouping trigger, thus we can define at which maximum distance these marks should be grouped. The above examples are simplified for the sake of explaining the core idea between grouping. In practice, marks are not perfectly straight lines, thus to extending them requires to estimate their direction, and also joining two marks requires smoothing to achieve seamless curvature of the line.
Figure 8. Grouping lines.
Path smoothing. After each stroke is complete, the system
automatically smoothens the line, in order to decrease the possible noise of the input information, as well to increase the appeal of the curves (see Figure 9). Most systems use a constant level of smoothing that the users can adjust according to their needs. However, our goal is to give the user direct control over the tools without interrupting the process of drawing. During sketching, the artists greatly vary the speed of applying strokes, naturally quick strokes tend to be smoother and vice versa. This variation in sketching speed can thus serve as an indicator of the required stroke smoothing.
Figure 9. Path smoothing.
Similarly to Thiel (2011), we use information about the input speed to regulate the level of applied smoothing to strokes. The smoothing is achieved by simplifying the path using an algorithm based on Schneider and Philip J (1990). The algorithm is supplied by PaperJS library, which accepts a simplification tolerance level. We have tested a range of simplifying levels on various path shapes, and concluded that shapes tend to get heavily distorted above 1000 level (see Figure 10). Thus we interpolate only between 0 and 1000 tolerance level. The speed level is counted as (path length / millisecond). The maximum speed we managed to achieve, by intentionally drawing as fast as possible, was 200 PaperJS distance units per ms. In a normal setting we expect the user to vary between 1 and 50 units per millisecond. The final level of smoothing counted as following:
Min( Min(50, speed / 50) * 1000 * smoothLevel, 1000)
where ‘smoothLevel’ is an optional level of the smoothing strength.
Figure 10. Simplification tolerance levels.
Inflection points detection. We count strokes as a line drawn
between the moment the digital pen touches the canvas and the moment it is released. Complex strokes are usually accompanied by pauses in hand movements at the inflection points, in order to adjust to changes in the direction of the stroke. While such an action would result in one continuous stroke, we semantically segment such strokes (see Figure 11) for several reasons. One reason is to treat them as separate strokes in order to apply smoothing to each segment independently. Another reason is to use inflection points to cleanup corners, since during movement pauses the hand can make subtle moves to inertia which results in undesirable corrections. A third reason is to give an increase in the
resolution of the undo stack, which can be useful in instances when the user draws a line in two segments, where the first segment has a pleasing result and the second has an undesirable shape. In such cases, the user would likely be willing to undo only the last segment of the drawing, rather than the whole line.
Figure 11. Corner detection.
Undo/Redo stack. We provide an option for users to enable an
automatic beautification mode. This would increase the chance of having an undesirable outcome, but would have the benefit of increased tracking. When this mode is enabled, our undo stack is keeping track of automatic transformations in order to give users the option to revert back to their original input. For example, if a line was not meant to be grouped, the transformation can be reverted back by pressing the letter X on keyboard.
4. IMPLEMENTATION
We choose to use web standards as the platform for our system, in order to make it widely accessible. The logic of the application is written in ECMA script (also known as JavaScript) which is a dynamic scripting language. For rendering graphics, we use HTML Canvas 2D Context http://www.w3.org/TR/2dcontext/ which can render vector graphics and geometric primitives. To save development time, we also used the high level library Paper.js http://paperjs.org/ which includes multiple abstractions such as scene management, smoothing, bezier curves, geometric calculations, etc.
5. EVALUATION
Our system was evaluated by 4 users, namely 2 professional digital artists and 2 amateurs. The purpose of this evaluation was to observe how SketchIron fits into various sketching workflows, and gather initial impressions and improvement suggestions from our participants. To a smaller extent, we also tried to assess how SketchIron’s algorithm compares against those of other tools. Because of the nature of this research, the evaluation was more of an explorative nature, rather than traditional user testing. A common workflow of our participants was based on using analogues tools or analogue-like tools. Participants were asked to output a sketch using vector tools of their choice (which was always Illustrator) and compare the process of creating a similar sketch using our system. We recorded videos of their drawing process and conducted interviews during and after their drawing process.
Participants were given a graphic tablet and a laptop (see Figure 13) and were assigned a task that was composed of the following steps:
Figure 12. User tasks examples.
Step 1. Participants were asked to sketch an artwork based on the
examples we provided to them. We created a collection of clean line images, which is the typical artwork in the vector domain. From the list of six examples, all participants ended up selecting one of the two images displayed in Figure 12.
Step 2. Participants were given a detailed tutorial of the
SketchIron and time to get accustomed with it.
Step 3. After participants felt comfortable using the system, they
were asked to iterate on the same concept they worked on in Step
1, but this time using our system. They were asked to replicate the
earlier sketch, with a reasonable extent of variation. The content they drew in this step had to be similar to the first step, but in order to avoid mechanical replication of the first step, we encouraged variations.
Step 4. We conducted a semi-structured interview focused on the
pros and cons of our system and obtained suggestions for possible improvements. During the drawing process participants were encouraged to make comments, which we were discussing in more details after the drawing session. The general structure of questions was as follows:
1. Did you achieve the intended results with SketchIron? 2.Were there things that made you struggle while working with SketchIron?
3. How does SketchIron compare to other tools?
4.Did you find the interaction with SketchIron ergonomic while sketching?
After the interviews, we evaluated the feedback and made small adjustments to the system based on that feedback. These adjustments did not change the fundamental concept described above, but rather improved small interaction details or included bug fixing.
5.1 SketchIron and sketching workflows
The overall feedback received from participants was that SketchIron integrates well with their sketching workflow and brings a relatively close resemblance to their analogue drawing process. This is in line with our initial assumptions for this research, i.e. that vector editing tools can be improved through various state-of-the-art techniques and thus better support the artists’ natural drawing workflow.
SketchIron received positive feedback towards the workflow of stroke correction and grouping. The automatic selection of the strokes based on proximity was regarded as intuitive and convenient by all users. We observed that the participants were using the correction mode rather freely and frequently, and we believe this occurred because the interaction was available to them at low cost of effort.
Another feature appreciated by our participants was that of gradual adjustments of overstrokes. This was found to resemble quite closely the analogue drawing process in which artists would overdraw their lines in order to correct them, especially due to the option to control the strength of the correction strokes. We observed that participants tended to use this feature more frequently after they got better familiarized with the tool and with this functionality in particular.
The use of dynamic levels of smoothing was also appreciated by all testers, and commonly referred to as “more natural” compared to constant smoothing values. The result of constant smoothing can often produce very geometrically-looking results which reduce the feeling of control and human touch. Since the smoothing is applied dependent on the speed of the drawing, and every participant had a different pace of drawing, the level of smoothing had to be adjusted manually to fit the preference of each user.
5.2 SketchIron vs other tools
Although we initially expected to observe a variety of preferences regarding vector editing tools among the participants, all of them reported that they only used Adobe Illustrator in their regular drawing workflows. The evaluation of SketchIron was thus done only in comparison with Illustrator, and no other tools were discussed during the evaluation process.
One area in which the participants preferred SketchIron over Illustrator was the stroke correction functionality. They expressed preference towards SketchIron’s automatic selection of strokes, as opposed to the more laborious task of having to explicitly select a stroke in Illustrator.
Another area in which SketchIron has proven to perform better than Illustrator was the stroke adjustment. In general, participants preferred SketchIron’s gradual adjustment approach as compared to Illustrator, which completely replaces overstrokes.
In the overall process of drawing, however, there were no strong preferences towards one tool. Illustrator received overall preference regarding functionality, while SketchIron was preferred in terms of the ease of interaction. During the test, we observed that Illustrator displays information about the path
construction which to some users appeared to be slightly distracting, since they were not interested in manipulating the anchor points.
The participants were asked to compare the final results between SketchIron and Illustrator. All participants preferred the results of SketchIron, commonly referring to it as “more natural looking”. On the other side, however, Illustrator can provide a more polished result when used in combination with tracing (see Figure 14). It is worth mentioning here that the Illustrator tracing option is more time-consuming. In the example presented in Figure 14, the process of analogue sketching took 3 minutes, the SketchIron version took 5 minutes, while the Illustrator tracing option took 14 minutes. We can thus conclude that SketchIron indeed resembles the analogue drawing process closer in terms of time and results, however Illustrator might be preferable if a polished result is needed.
Figure 14. Sketching results example.
5.3 Improvement suggestions
Participants managed to achieve the overall intended results using SketchIron, however there were moments when the results of some interactions did not entirely correspond to the user’s intentions, or some expected features were missing from the tool. A common obstacle while using SketchIron was the lack of an eraser tool. While users could correct strokes, in some cases there was a need to erase entire fragments of the composition. The lack of the eraser puzzled many participants and all of them expressed the strong need for this feature. Besides the lack of an eraser tool, some expressed the need for other features which are typically present in vector drawing software. We believe that our minimum set of features in SketchIron was enough to serve the purpose, except for eraser tool which turned out to be essential in the process of sketching. In this regard, Illustrator was more comforting to users, since they could achieve some of their goals in a more straightforward or familiar way.
Another area which could be improved according to our participants was the overstrokes adjustment.Although they found the gradual adjustment useful,a few participants mentioned that in particular cases they would prefer to avoid this feature. Together with participants, we concluded that the use of pressure sensitivity to control the level of interpolation between the original and correcting stroke would solve these issues and would generally be a useful feature. The pressure sensitivity feature was not provided in SketchIron due to technical limitations.
Another common comment was that the beautification features did not always behave as expected. This is a common constrain of
such systems, due to the ambiguous nature of the input gestures. The common cause of confusion was the mark grouping algorithm where two strokes can be combined in multiple ways (see Figure 15). Participants used the undo feature to correct such gestures. In several instances, we observed that some participants started to develop intuition in predicting such cases. We thus concluded that using the software for longer period of time would develop intuition that would help reduce frequency of such cases occurring. We believe that this imperfection was one of the major source of dissatisfaction with SketchIron.
Figure 15. Stroke grouping outcome possibilities.
All tasks that were given to participants were oriented on clean and continuous lines. One of the participant expressed interest in trying out SketchIron in drawing something without using correction and mark grouping, but instead using only simple lines as on paper or analogue like tools (see Figure 16). We were pleased to conclude that SketchIron accommodates well such a drawing style, and both the participants and we liked the results. By implementing simple user controls and removing information about vector paths, we facilitated an analogue-like drawing experience. This gives flexibility to accommodate different sketching styles. Potentially we could use pressure sensitive information to obtain opacity or line thickness to closer simulate an analogue-like environment.
Figure 16. Drawing without corrections.
6. LIMITATIONS AND FUTURE WORK
A common problem of path correction and beautification systems is the uncertainty of the user input, which might lead to unexpected results. The more information the user can provide, the more relevant the results would be. While sketching, it is important to keep the user focused on his task and to avoid interrupting the creative process. We see the opportunity in obtaining more implicit user input from various sources. For example, we are using the speed of drawing as a parameter for smoothing levels, however, additionally to the speed, we could also use pressure sensitivity. As we observed in our testing, the erasing features are essential to the sketching process. Using the underlying constructions of the vector paths, it is possible to derive useful information of path segmentation and to provide intelligent gestures for smart erasing. For example, it might be useful to segment parts based on their intersections points, in order to provide quick ways to erase fragments of the path. Due to lack of performance optimizations, the resolution of our underlying data structures is quite low, which resulted in unwanted artifacts when working with small scale strokes. Optimizing the resolution of paths would improve the quality of the transformations which would lead to a better user experience, especially in cases when working with short strokes. Another concern is the pooling rate of the input device (the resolution of the input) since our application is running in a web browser, it is not known to us if the sampling of the user input is done at the full capacity of the hardware.
There is a variety of potentially useful gestures that could be communicated using multitouch technologies. While multitouch is out of the scope in this paper, we believe that there are many opportunities in using finger gestures, due to fact that the user can provide more semantics to his actions while sketching.
Another limitation is linked to the evaluation process. In our experiments, we used a pen tablet, which for drawing purposes is supreme compared to a mouse, but not as good as direct manipulation tools such as pencil and paper or tablet with a screen. The majority of our participants were not advanced users of pen tablets, which contributed to friction in their process. We believe that the overall experience of the experiment would have been better using a graphic tablet, however the tool itself would not solve all the software design challenges.
7. CONCLUSIONS
The mathematical nature of vector artworks demands precision and effort from the user input which makes the process of sketching tedious and distractive. In this paper, we demonstrated interaction design techniques of vector manipulation tools which better accommodate the sketching process. We took inspiration from interaction techniques that are present in various systems, as well as techniques described in academic papers. Our contribution was to combine these interaction techniques into a workflow that would better support the sketching process in the context of vector drawing.
During the testing of our system, the users appreciated the use of implicit interactions which reduce the friction between the sketching process and the controlling modes of the tools. The feature of correcting strokes without selecting a target stroke
allowed to maintain a fluid transition between drawing and correction mode. The use of drawing speed to vary the level of smoothing gave users control over stroke beautification without the need to explicitly configure it. Some of the users (depending on their drawing style) appreciated the incremental approach to stroke correction, which requires less precision and supports gradual adjustments. While vector-based sketching is not a popular choice among users today, we believe that vector-based tools can be a productive medium for sketching. As we demonstrated in this paper, reducing the orchestration of the vector tools could provide a more fluid sketching experience. Given the time constrains and resources, the proposed system only covers a fraction of the features needed to accommodate a complete sketching workflow. In our studies, we compared our system with the state of art software, and received positive feedback regarding our overall direction. With the upcoming trends of multitouch technologies, we believe that vector drawing software can greatly benefit from the use of direct manipulation techniques.
8. REFERENCES
Olsen, L., Samavati, F., Sousa, M. C., & Jorge, J. (2008, April). A taxonomy of modeling techniques using sketch-based interfaces. In Eurographics State of the Art Report.
Jorge, Joaquim, and Faramarz Samavati, eds. Sketch-based
interfaces and modeling. Springer Science & Business Media,
2010.
Grimm, C. (2011). Results of an observational study on sketching.
Report Number: WUCSE-2011-57 (2011). In All Computer Science and Engineering Research. http://openscholarship.wustl.edu/cse_research/62
McCrae, J., & Singh, K. (2011, August). Neatening sketched strokes using piecewise French curves. In Proceedings of the
Eighth Eurographics Symposium on Sketch-Based Interfaces and Modeling (pp. 141-148). ACM.
Frisken, Sarah F. "Efficient curve fitting." Journal of Graphics,
GPU, and Game Tools 13.2 (2008): 37-54.
Brandt, C., Seidel, H. P., & Hildebrandt, K. (2015, May). Optimal Spline Approximation via ℓ0-Minimization. In Computer
Graphics Forum (Vol. 34, No. 2, pp. 617-626).
Baran, I., Lehtinen, J., & Popović, J. (2010, May). Sketching clothoid splines using shortest paths. In Computer Graphics
Forum (Vol. 29, No. 2, pp. 655-664). Blackwell Publishing Ltd.
Thiel, Y., Singh, K., & Balakrishnan, R. (2011, October). Elasticurves: Exploiting stroke dynamics and inertia for the real-time neatening of sketched 2d curves. In Proceedings of the 24th
annual ACM symposium on User interface software and technology (pp. 383-392). ACM.
Arvo, J., & Novins, K. (2000, November). Fluid sketches: continuous recognition and morphing of simple hand-drawn shapes. In Proceedings of the 13th annual ACM symposium on
User interface software and technology(pp. 73-80). ACM.
Zeleznik, Robert C., et al. "Lineogrammer: creating diagrams by drawing."Proceedings of the 21st annual ACM symposium on
User interface software and technology. ACM, 2008.
Wang, Beirong, Jian Sun, and Beryl Plimmer. "Exploring sketch beautification techniques." Proceedings of the 6th ACM SIGCHI
New Zealand chapter's international conference on Computer-human interaction: making CHI natural. ACM, 2005.
Cheema, S., Gulwani, S., & LaViola, J. (2012, May). QuickDraw: improving drawing experience for geometric diagrams. In Proceedings of the SIGCHI Conference on Human Factors in
Computing Systems (pp. 1037-1064). ACM.
Paulson, B., & Hammond, T. (2008, January). PaleoSketch: accurate primitive sketch recognition and beautification. In Proceedings of the 13th international conference on Intelligent
user interfaces (pp. 1-10). ACM.
Igarashi, T., Matsuoka, S., Kawachiya, S., & Tanaka, H. (1997, October). Interactive beautification: a technique for rapid geometric design. InProceedings of the 10th annual ACM
symposium on User interface software and technology (pp.
105-114). ACM.
Fišer, J., Asente, P., & Sýkora, D. (2015, June). ShipShape: a drawing beautification assistant. In Proceedings of the workshop
on Sketch-Based Interfaces and Modeling (pp. 49-57).
Eurographics Association.
Pavlidis, T., & Van Wyk, C. J. (1985, July). An automatic beautifier for drawings and illustrations. In ACM SIGGRAPH
Computer Graphics (Vol. 19, No. 3, pp. 225-234). ACM.
Murugappan, S., Sellamani, S., & Ramani, K. (2009, August). Towards beautification of freehand sketches using suggestions. In Proceedings of the 6th Eurographics Symposium on
Sketch-Based Interfaces and Modeling (pp. 69-76). ACM.
Wang, S., Qin, S., & Gao, M. (2014). New grouping and fitting methods for interactive overtraced sketches. The Visual
Computer, 30(3), 285-297.
Pusch, R., Samavati, F., Nasri, A., & Wyvill, B. (2007). Improving the sketch-based interface. The Visual Computer, 23(9-11), 955-962.
Barla, P., Thollot, J., & Sillion, F. X. (2005, July). Geometric clustering for line drawing simplification. In ACM SIGGRAPH
Orbay, Günay, and Levent Burak Kara. "Beautification of design sketches using trainable stroke clustering and curve fitting." Visualization and Computer Graphics, IEEE Transactions on 17.5 (2011): 694-708.
Baudel, T. (1994, November). A mark-based interaction paradigm for free-hand drawing. In Proceedings of the 7th annual ACM
symposium on User interface software and technology (pp.
185-192). ACM.
Henzen, A., Ailenei, N., Fiore, F. D., Van Reeth, F., & Patterson, J. (2005, November). Sketching with a low-latency electronic ink drawing tablet. InProceedings of the 3rd international conference
on Computer graphics and interactive techniques in Australasia and South East Asia (pp. 51-60). ACM.
Schneider, P. J. (1990, August). An algorithm for automatically fitting digitized curves. In Graphics gems (pp. 612-626). Academic Press Professional, Inc..
