A Graph-Based Representation Method for Fashion Color

Abstract

Fashion color research takes the color information of fashion apparel as the major focus for further studies, such as style categorization or trend prediction. However, the colors in apparel are treated as isolated elements from each other, disregarding the fact that not only the attributes of each color itself but also the collocation relationship of the colors in apparel are important color factors. To provide a more comprehensive abstraction of the information from the fashion colors as well as emulating the human cognition of fashion colors, in this paper, we are the first to propose a knowledge graph-based representation method that captures not only the individual colors but also abstracts the spatial relation of all the colors that appear in a single piece of fashion apparel. This method provides the fundamental definition of the abstraction of the relation of colors, a detailed method to construct the color graph, as well as the practical matrix-based management and the visualization of the constructed graphs. The case studies for color data extraction and extended usage demonstrate the effectiveness of our method with comprehensive color data representation and effective information extraction.

1. Introduction

Fashion trends are notoriously volatile and change quickly with time. One major channel to determine fashion trends are fashion shows [1], where the latest fashion designs are debuted. In this event, fashion designers showcase their upcoming lines of clothing through models. According to previous research, color, cut, head decorations, and pattern are four major elements to investigate in fashion styles [2]. The color of the fashion apparel plays a pivotal role in fashion trend prediction.

However, in most of the existing fashion color research, such as fashion color trend prediction [3,4,5] or fashion color compatibility study [6,7], each of the colors in a piece of fashion apparel is extracted and quantified as an isolated element, as shown in Figure 1. Even if there are multiple colors in the same apparel item, each color is still being measured and quantified independently in the collection and representation of the color data.

Although there are some fixed composition principles for color collocation, they are still not enough to reflect human perception of the colors from a given piece of fashion apparel. Human perception of color is a comprehensive process including not only the attributes of each color itself, but also the collocation relationship between colors [9]. However, there is no work that studies the proper relationship of the colors in fashion apparel with the comprehensive extractable information.

To capture the relationship between the colors in fashion apparel, there is a need for the proper representation of the color data from a single fashion apparel item as well as extraction of comprehensive information about it. The popularity of the knowledge graph has demonstrated the effectiveness of representing real world entities, i.e., objects, events, situations, and concepts, with comprehensive representation of the relationships between them [10,11,12,13].

For the reasons above, we aim to develop an advanced color representation of a given fashion apparel item that provides more information about the color data, including not only the attributes of individual color elements, but also their collocation, matching, and geographical information. Given a piece of fashion apparel, four consecutive problems are studied with our method: (1) the number of colors; (2) the ratios of each color; (3) the relationships between each of the colors; and (4) the representation of the colors for better perception.

This paper studies advanced color representation for fashion apparel with the expectation of understanding whether we can better grasp and recognize the design information of the colors by establishing a more comprehensive color representation from a given fashion apparel item. These results are of great significance for fashion color research. In addition, they also provide a new method for computer-aided quantitative analysis for the recognition and style matching of colors in fashion apparel. In modern color trend prediction and analysis with big amounts of data input, especially when using the most popular machine learning methods, machines are still trying to mimic human cognitive processes for analysis and prediction; however, human cognition of color has not been fully understood and studied. Therefore, the study in this work also aims to add the power to the exploration of color cognitive study.

The rest of the paper is organized as follows. Section 2 introduces the background and our motivations. Section 3 presents our proposed graph-based color representation method. We evaluate our proposed methods with different case studies in Section 4. Section 5 concludes the paper.

2. Background and Motivation

The study in this work relies on the fundamental color quantification for fashion apparel and the basics of knowledge graphing. Before we present our proposed method, we first investigate the conventional fashion color quantification and the concept of knowledge graph.

2.1. Fashion Color Quantification

Conventional fashion color quantification includes analyzing and interpreting colors or patterns through spatial information in an image of fashion apparel or adopting statistical methods to analyze the color of fashion apparel items, as shown in Figure 1. The color information about a given piece of apparel is extracted with the required measurement method and quantified to a numerical representation, e.g., the type and number of the colors or the ratios of different colors in a pie chart.

As an instance of the former category, Kang et al. [14] studied the color cycle to analyze patterns on fabrics; Wu et al. predicted the popularity of blue and the color phase tendency with the help of the big data processing capability of computers by analyzing the distribution of color imagery and color phase tendency of a collection of women’s clothing for spring and summer [15].

Representative methods in the latter category are Chang et al., who used ring histogram to study the pattern of colors together with an edge growth algorithm to separate the colors from each other; Chio et al., who proposed a color histogram search for clothes in the same category [16]; Wang et al., who used color matching and pattern matching to search clothes in the photographs of customers [17]; and Chen et al., who proposed a learnable pattern classifier to learn and classify the semantic features of the colors of clothes [18].

As a conclusion, the conventional methods for color quantization or color analysis only involve the type of colors, ratio of each color, color cycle, or color phase tendency, none of which show the relationships between each of the colors in the same piece of fashion apparel. The color data from the methods above are treated as isolated instances, that is, isolated quantifiable arguments or objects. Even with the modern learning-based methods [3,19,20], the direct relationships between colors are left to the black box models and are still ignored and left absent from the input. The absence of knowledge on the relationships between the colors and the way humans perceive images of fashion apparel motivated us to study the proposed method in this paper, which attempts to directly abstract and further utilize the comprehensive information from the colors in a piece of fashion.

2.2. Knowledge Graph

A graph is a non-linear data structure consisting of nodes and edges as shown in the Figure 2, where the nodes only contain a single integer value. The nodes are sometimes also referred to as vertices and the edges are lines or arcs that connect any two nodes in the graph. Formally, a graph is defined as a finite set of vertices (or nodes) and set of edges which connect a pair of nodes.

Graphs are de facto data structures used to represent the relationships between entities in many application domains, such as social networks, genomics, and neuron models. A graph representation is more suitable to the modelling of fashion outfits. Each graph is an ensemble of nodes and their interactions (i.e., edges), thus offering a natural abstraction of the compatibility among items in a fashion outfit. Graphing is adopted to represent the pairwise item compatibility in [21], where it was used to predict the compatibility of different pairs of fashion apparel. Graphs are also used to present the outfits in the work of Cui et al. [21]. Both of them took a straightforward approach to modeling each item as a node in the graphs and fed such graphs into graph neural networks (GNN) or graph convolutional networks (GCN) for feature extraction [3,19]. However, their works focus on an overall compatibility extraction of the fashion outfits instead of studying the compatibility of colors inside a single item or a whole outfit, missing the in-depth understanding of the relationships between the different colors.

Instead of modeling fashion items as nodes in a graph, in this paper, we change our focus to the colors in a single piece of fashion apparel. We model each color item and the area occupied by it in a piece of apparel as a node, and the geometrical location of the colors as the edge to fully leverage the representation capability of graph data structure. This graph structure of the colors is more suitable to understanding the utilization of the colors in a design as well as the color patterns than the original quantification methods.

3. Graph Representation of Fashion Color

Building a graph for fashion apparel involves a series of process including extracting the categories of the colors, analyzing the ratio of the colors to the overall visual presentation, defining and analyzing the geographical location of the different color items, and building up the graph of the colors for the chosen object, as shown in Figure 3.

To present the processes step-by-step, we first take the regularly shaped objects as the examples to explain our approaches. We then further extend the approach to illustrate our focused fashion outfits.

3.1. Color Clustering and Ratio Calculation

The basic information of the colors in fashion apparel is the number of distinguishable colors and the ratio of each of them in presence. While conventional methods have already provided sufficient solutions to extract this information, we adopt some mature ones in our method.

3.1.1. Color Clustering

As shown in Figure 4, images of regular objects with different colors can be easily recognized, however, as digital images originally contain inaccurate sensing of the original colors, we obtained the most popular and error-proof k-means-based color clustering method for the input color classification and enumeration. The resolution of the k-means-based methods can be different, but we adapted the settings to capture the most significant colors based on our experimental results. As shown in the first data column in Figure 4, the colors are first clustered and then enumerated. The names and numbers of the colors in the brackets indicate the color clusters and the number of colors.

3.1.2. Color Ratio Calculation

The ratios of the colors in the image are calculated based on the number of pixels, where the pixels’ values approximated to the color cluster are considered to be effective. After the calculation of the pixels, all the colors collected are normalized as a percentage, since the absolute pixel number is not important in our method, as shown in the second column in Figure 4, where the ratio of each color is calculated in regard to the colors clustered in the previous color enumeration stage.

3.2. Color Graph Construction

Except the colors and the ratios of them, the style of a garment’s colors is mainly represented by their situational combination, which without advanced collection of the information, is hard to capture. Hence, as discussed above, we use a graph to represent both the conventional information (colors and their ratios) and the locational information. The graph with nodes and edges is noted as Equation (1).

$$G=\left(V,E\right)$$

(1)

where V represents the node set (or called vertices), and E represents the edge set.

3.2.1. Node Definition

To build the graph of the color data for an object, we first define each node in V to represent the enumerated color categories and the ratio of them in the form of (color, ratio) for each of the colors appearing in an object. For usage without the consideration of the ratio of the colors, a node can also be in the form of only the color information as (color) for simplicity. We define the edges of the graph to represent the geographical location of the colors collected.

3.2.2. Geographical Information

The geographical information is another important but missing category of information for color data extraction in the conventional solutions. The geographical information is of great importance to the human perception of overall color style. Here, geographical data are collected based on the visual relationships of the different colors and are built as the edge set E. To quantitatively represent the relationships and keep the logic concise, we abstracted the definition of the relationships into the following three categories:

• Discrete: where two colors are nonadjacent. As shown in Figure 4b,c, the red area and yellow area in Figure 4b and red area and purple area in Figure 4c are discrete based on our definition.
• Adjacent: where two colors share the same boundary but one of them is not surrounded by the other. As shown in Figure 4a–c. The red and blue in the objects are all adjacent colors.
• Surround: where one color is in the other color and the pixels of the inner color are all surrounded by the outer one. As shown in Figure 4d, the red color is surrounded by the blue color.

We used a directed graph to capture the geographical information of the nodes, since it can show the surrounded status of the colors—the adjacent colors have a bi-directional edge, and the surrounded colors have a single direction of the edge. As shown in Figure 5a, the red and blue colors are adjacent and blue and yellow are also adjacent, so these two pairs both have a bi-directional edge to each other, while the red and yellow are discrete, so they do not have an edge between them.

Figure 5b depicts a case in which a red color is surrounded by a blue color, so they have a single direction of edge that points from the one that is surrounded to the one that surrounds it. In this way, the number of colors, the ratio of each color, and the relationship between the colors are captured and represented as a graph. Although these examples are for regular images with simple patterns, the method can be used for fashion apparel with complex color patterns and irregular shapes due to the ignorance of shape and the scalability of the number of colors in our solution.

3.3. Color Graph Visualization and Management

With the defined vertex (node) and edge, we represent the color graph of a given object with the vertex and edge pairs as shown in Figure 5. The vertices set contains the color and its ratio, and the edges set contains the geographical relationships between each of them. This enables the visualization of the original color with a set of nodes connected through edges. Note here that the ratio of the color is being represented as a weight of the vertices in the graph visualization to present the captured information. In this way, a more quantitative extraction of the color information in an object is obtained and visualized.

To simplify the visualization and management of the extracted color information, we store the extracted V and E as two matrices, as shown in the example in Figure 6. Due to the contained information, both matrices are two-dimensional matrices in which the vertex matrix includes the ratios of the colors, and the edges matrix stores the relationships of all the different color pairs. In this way, all the extracted data are properly stored and we can easily retrieve the matrices to extract the color information.

4. Evaluations

To demonstrate the effectiveness of our proposed graph-based fashion color representation method, we first adapted our proposed method to two selected real-world fashion outfits, where the color graphs are extracted and presented. Furthermore, we show the capability of our proposed method for complex fashion color representation which can be used to distinguish between designs that cannot otherwise be differed with the conventional color extraction method.

4.1. Color Graph Extraction for Fashion Outfit

Different from regularly shaped objects, fashion apparel has irregular shape and irregular locations of the colors; however, our geographical definition is adaptable to irregular color patterns as well. In this evaluation, we demonstrate the effectiveness of our solution for real-world fashion apparel. As shown in Figure 7 for two real fashion outfits, the extraction of the color, ratio, and relationship information is shown in the different steps. Note here that since we focus on the fundamental exploration of the color instead of the patterns of the other designs used in the examined fashion outfits, we use fashion outfits with significant colors as our target. To simplify the graph, the same color at different locations is demonstrated as in one single vertex with the accumulated ratio of all the areas clustered as the same color.

As we can see from Figure 6, the vertices, which represent the category and ratio of the colors, are extracted by the k-means algorithm-based method, and the ratios of the different colors are calculated based on the pixels in each of the color clusters. The color graphs of different fashion outfits show dramatic differences from each other, but the number of colors, the ratio of them, and the color adoption style is captured by the graph structure, which is never shown by conventional methods. These demonstrate the effectiveness of our method compared to the conventional color representations due to its more comprehensive color information. The extracted color graphs also show the significant differences between the color adoption and color style regarding the used colors and the relationships between them, which could be used for further analysis of the color pattern and design style study.

4.2. Extended Adoption of the Proposed Method

To show the effectiveness of our graph-based color representation method, we further evaluate it with two sets of fashion outfits: (1) fashion outfits with simple color patterns and (2) fashion outfits with complex color pattern. As shown in Figure 8 and Figure 9, with the conventional color representation, the two sets of fashion outfits are represented by the same pie chart, since the pie chart-based method only focus on the categories of colors and the ratios of them to the whole design. However, with our graph-based representation method, the relationship of the different colors is clearly abstracted, e.g., the red and white colors are adjacent to each other in the first sample in Figure 8, while the red color is surrounded by the white color in the second sample; the geographical relationship is clearly abstracted in our graph-based representation method with the involvement of a pre-defined edge.

In addition, Figure 9 demonstrates the effectiveness of our solution for fashion outfits with complex color patterns. As shown in the figure, the ratios of the colors are the same although some of them are adjacent while some of them are distributed across the whole design. The additional edges in the graph representation for the first sample clearly shows the adjacent relation of the (blue, black) and (green, black) color sets, while the blue and green colors are discrete to each other, but both are adjacent to the white color. However, in the latter one, the black color is considered as adjacent to the white since some of its edges are not in the white color area while the blue and green are considered as surrounded by white color. The color graphs clearly capture the categories, ratios and relationships of the colors. These two sets of evaluations demonstrate the advanced information extraction of our graph-based color representation method towards the conventional methods that do not consider the relationships of the different colors appearing in fashion apparel.

5. Conclusions and Future Work

Fashion color is an inevitable factor for fashion trend prediction. In this paper, we address an important problem which is the absence of the geographical relationships of colors in fashion apparel. We propose a color graph which is based on the knowledge graph to capture and represent the colors. It involves in a comprehensive process including color data extraction, relationship definition and analysis, color graph construction, and graph data management. The evaluations show that our proposed method is effective for color data capturing and representation with more comprehensive information. Currently, we do not consider complex patterns such as embellishment in this initial design, it will be explored in future work with more complex graph definitions. Our method could potentially provide more insights into human perception of the color information of a fashion design and provide a new way of color data mining for future computer-aided color research on fashion apparel.