How to Use Graph Neural Networks for Text Classification?

The graph neural networks are trending because of their applications in a variety of predictive analytics tasks. When it comes to modelling the data available with graphical representations, graph neural networks outperform other machine learning or Deep learning algorithms. In the field of natural language processing as well, graph neural networks are being applied in a full swing because of their capabilities to model complex text representations. In this article, we will discuss one such interesting application of graph neural networks, i.e., in text classification. First, we will understand how this framework works to model the text representations and then we will explore how it can be used for text classification. The major points that we will discuss in this article are listed below.

As we know that we can roughly divide the Deep learning studies into two major models one is convolutional neural networks and another one is recurrent neural networks. If we talk about the text classification the studies here also can be divided into two groups where one is focused on the make model which can learn based on the word embedding. Three are various studies that have shown us that the success of any text classification model depends on the effectiveness of the word embeddings. Another group is focused on learning the document and the word embedding together.   

If we talk about the models CNN and RNN both can be used for text classification. But the CNN is good with the one-dimensional convolutional and is majorly used in the computer vision field and a special type of RNN that is LSTM (long short term memory) models can be used for better performance in the text classification. The further extension of these models is done using mechanisms, like attention mechanisms which have increased the flexibility of text representation and can be used in the deep learning model as an integral part. Deep learning methods and their extensions are widely used. The major benefit of these models is they focus on the local consecutive word sequence for text classification, unlike traditional methods which use global co-occurrence information of the words for learning saved in a corpus. 

In recent times the applications of graph neural networks are growing rapidly in multiple domains. There are a number of studies in which we can see that number neural networks such as CNN are generalized and can be applied to the regular grid structure which is helpful in working on arbitrarily structured graphs. There are multiple Graph Neural Networks in the field of text classification. Before going for any of the networks let us understand the graph neural networks first which will make a clearer picture of the network in front of us. 

The graph is a data structure that consists of vertices and edges. It can be represented as the function of vertices and edges.

The below image is a representation of the direct graph.

The type of the edges in the graph de[ends on the directional dependencies between the vertices. And the edges can be of two types: directed or undirected.

By the name, we can understand if a neural network operates on the graph we can call it a graph neural network where the major operation of any neural network is to classify the vertices or nodes. So that every node presented in the graph can be classified by their provided labels according to the neural network. 

If there is a node v and its features can be characterized by the x_v at ground truth t_v given in a labelled graph G so we can label an unlabeled graph using this labelled graph where a d dimensional vector helps in learning and h-v contains the information of its neighbourhood. Mathematically 

If any reader wants to learn more about graphical neural networks can check this paper.

As we have discussed before in the deep learning section, we can perform text classification using CNN or RNN. and also how LSTM is good for text classification we can check on this article.