Edge-Guided Automatic Image Completion: A Novel AI Illustrator Finishing Your Incomplete Sketches

Overview

Drawing well-balanced pictures from scratch entails both plenty of time and proficient skills. In this project, I devised a novel deep-learning-based system, which provides automatic completion of users' unfinished drawings. The system was designed to suggest possible endings of users' current sketches interactively so that even novice users can easily draw their ideal images by referring to these suggestions. Although I collected only complete images to build the dataset, the model was able to learn to create new plausible images from partial freehand sketches. To the best of my knowledge, this is the first attempt to create complex images from users' unfinished sketches. For example, sketch-rnn can complete users' strokes, but the final outputs are limited to simple stroke-based objects. In this work, I investigated a new method of synthesizing more complicated images such as 'Anime Characters' from a few sparse strokes. In the preprocessing phase, with my proposed algorithm, the training images were converted into partial contours that resemble human's hand-drawn sketches. Then, the completion process was performed through two consecutive stages, 1) contour completion, 2) colorization and refinement. Assigning a different model to each stage reduced blurriness and artifacts. Additionally, I carefully considered network architectures, loss functions, and other parameters to avoid mode collapse as the network often disregarded the variety of produced images. This proposed method enables us to intuitively control features of generated images by outlines without any other user's guidance.

Competition history

• ISEF 2020 Robotics and Intelligent Machines · Entry ROBO028

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.