The Mind of Pixels: Video Aesthetic Feature Analysis Based on Graph Representation of Pixel Block Relations

Overview

There are uncountable videos, but do popular videos follow a computable pattern? Computational aesthetics have been developed in the medium of single images. But in comparison, video generates more complex data in terms of space and time. This paper analyzes videos' aesthetic features based on graph learning of pixel block relations. Each pixel block’s intensity and entropy are computed to construct a graph of each video frame. By analyzing the influence of each vertex on the graph structure of video frames, this paper defined the Pixel Block Participation Rate (PBPR) as the measure of independence and prominence of pixel blocks. The method is further studied on two datasets. On the small dataset of 8 movie trailers, composition styles like “Centered Layout”, “Left-wing Structure”, “Sunken Structure”, and others were found. Correlations were revealed in the spatial layout of pixel blocks between the video frames. On the big dataset of the 1000 most popular short videos on bilibili – a video-sharing website, Moran’s I is used to investigate the spatial clustering of the PBPR. The Computational Aesthetic Measure of Videos is defined. It is found that the videos in the dataset followed six representative composition patterns. This research integrated the proposed aesthetic feature analysis algorithms in the paper. This visualization framework can extract the composition patterns which videos of different genres follow. This research enriches computational aesthetics by introducing a new method of measuring the aesthetics of videos by PBPR. It can further help with the abstraction of video features for deep learning.

Competition history

• ISEF 2021 Robotics and Intelligent Machines · Entry ROBO027

Resources

• ISEF abstract page ↗

Related projects

Closest projects by meaning, across every fair and year in the corpus.