Abstracts:Network embedding aims to represent nodes with low dimensional vectors while preserving structural information. It has been recently shown that many popular network embedding methods can be transformed into matrix factorization problems. In this paper, we propose the unifying framework “Z-NetMF,” which generalizes random walk samplers to Z-Laplacian graph filters, leading to embedding algorithms with interpretable parameters. In particular, by controlling biases in the time domain, we propose the Z-NetMF-t algorithm, making it possible to scale contributions of random walks of different length. Inspired by node2vec, we design the Z-NetMF-g algorithm, capturing the random walk biases in the graph domain. Moreover, we evaluate the effect of the bias parameters based on node classification and link prediction tasks. The results show that our algorithms, especially the combined model Z-NetMF-gt with biases in both domains, outperform the state-of-art methods while providing interpretable insights at the same time. Finally, we discuss future directions of the Z-NetMF framework.

Abstracts:Pedestrian trajectory prediction has been broadly applied in video surveillance and autonomous driving. Most of the current trajectory prediction approaches are committed to improving the prediction accuracy. However, these works remain drawbacks in several aspects, complex interaction modeling among pedestrians, the interactions between pedestrians and environment and the multimodality of pedestrian trajectories. To address the above issues, we propose one new trajectory distribution aware graph convolutional network to improve trajectory prediction performance. First, we propose a novel directed graph and combine multi-head self-attention and graph convolution to capture the spatial interactions. Then, to capture the interactions between pedestrian and environment, we construct a trajectory heatmap, which can reflect the walkable area of the scene and the motion trends of the pedestrian in the scene. Besides, we devise one trajectory distribution-aware module to perceive the distribution information of pedestrian trajectory, aiming at providing rich trajectory information for multi-modal trajectory prediction. Experimental results validate the proposed model can achieve superior trajectory prediction accuracy on the ETH & UCY, SSD, and NBA datasets in terms of both the final displacement error and average displacement error metrics.

Abstracts:Recent studies have revealed that GNNs are vulnerable to adversarial attacks. Most existing robust graph learning methods measure model robustness based on label information, rendering them infeasible when label information is not available. A straightforward direction is to employ the widely used Infomax technique from typical Unsupervised Graph Representation Learning (UGRL) to learn robust unsupervised representations. Nonetheless, directly transplanting the Infomax technique from typical UGRL to robust UGRL may involve a biased assumption. In light of the limitation of Infomax, we propose a novel unbiased robust UGRL method called Robust Graph Information Bottleneck (RGIB), which is grounded in the Information Bottleneck (IB) principle. Our RGIB attempts to learn robust node representations against adversarial perturbations by preserving the original information in the benign graph while eliminating the adversarial information in the adversarial graph. There are mainly two challenges to optimizing RGIB: 1) high complexity of adversarial attack to perturb node features and graph structure jointly in the training procedure; 2) mutual information estimation upon adversarially attacked graphs. To tackle these problems, we further propose an efficient adversarial training strategy with only feature perturbations and an effective mutual information estimator with the subgraph-level summary. Moreover, we theoretically establish a connection between our proposed RGIB and the robustness of downstream classifiers, revealing that RGIB can provide a lower bound on the adversarial risk of downstream classifiers. Extensive experiments over several benchmarks and downstream tasks demonstrate the effectiveness and superiority of our proposed method.

Abstracts:With the explosive growth of scholarly information, researchers spend much time and effort copiously quoting authoritative works to support their ideas or motivations. We aim to alleviate this situation by proposing a citation recommendation strategy that recalls related papers for a rough idea (a piece of text, i.e., abstract, manuscript). However, the perspective of existing citation recommendations can not be well applied to our task for two defects. First, these methods neglect the reasoning of research topics, which makes the recommendation mechanism not meticulous enough and lacks explainability. For instance, they are not able to mine the hidden citing logic for the candidate paper while recommending. We fill the research gap by constructing structural topics consisting of knowledge concepts from the textual content, where reasoning paths between topics are extracted from an external knowledge graph. Second, the citation network is viewed as a crucial structural context to enhance the recommendation performance, but the new target idea does not have links to the citation network as published papers do. To simulate the prospective topological structure, our model, meanwhile, incorporates a contrastive-learning-based alignment paradigm to encourage the consistency of content embeddings and structure-oriented embeddings. We evaluate our proposed model on three real-world datasets and demonstrate that it significantly improves recommendation accuracy while providing high-quality knowledge-aware reasoning. And an interesting visual example illustrates the reasoning process when our model actually judges samples, which supports the feasibility of our topic-view learning paradigm.

Abstracts:Weather forecasting is of great importance for human life and various real-world fields, e.g., traffic prediction, agricultural production, and tourist industry. Existing methods can be roughly divided into two categories: theory-driven (e.g., numerical weather prediction (NWP)) and data-driven methods. Theory-driven methods require a complex simulation of the physical evolution process in the atmosphere model using supercomputers, while most data-driven methods learn the underlying laws from the historical weather records via deep learning models. However, some data-driven methods simply regard all weather variables of monitoring stations as a whole and fail to more granularly exploit complex correlations across different stations, while others prefer to construct large neural networks with massive learnable parameters. To alleviate these defects, we propose a spatio-temporal contrastive self-supervision method and a generative contextual self-supervised technique to capture spatial and temporal dependencies from the station-level and variable-level, respectively. Through these well-designed self-supervised tasks, uncomplicated networks obtain strong capability to capture latent representations for weather changes with time-varying. Thereafter, an effective encoder-decoder based fine-tuning framework is proposed, consisting of three self-supervised encoders. Extensive experiments conducted on four real-world weather condition datasets demonstrate that our method outperforms the state-of-the-art models and also empirically validates the feasibility of each self-supervised task.

Abstracts:As the study of graph neural networks becomes more intensive and comprehensive, their robustness and security have received great research interest. The existing global attack methods treat all nodes in the graph as their attack targets. Although existing methods have achieved excellent results, there is still considerable space for improvement. The key problem is that the current approaches rigidly follow the definition of global attacks. They ignore an important issue, i.e., different nodes have different robustness and are not equally resilient to attacks. From a global attacker's view, we should arrange the attack budget wisely, rather than wasting them on highly robust nodes. To this end, we propose a totally new method named partial graph attack (PGA), which selects the vulnerable nodes as attack targets. First, to select the vulnerable items, we propose a hierarchical target selection policy, which allows attackers to only focus on easy-to-attack nodes. Then, we propose a cost-effective anchor-picking policy to pick the most promising anchors for adding or removing edges, and a more aggressive iterative greedy-based attack method to perform more efficient attacks. Extensive experimental results demonstrate that PGA can achieve significant improvements in both attack effect and attack efficiency compared to existing graph global attack methods.

Abstracts:Semi-supervised graph learning aims to improve learning performance by leveraging unlabeled nodes. Typically, it can be approached in two different ways, including predictive representation learning (PRL) where unlabeled data provide clues on input distribution and label-dependent regularization (LDR) which smooths the output distribution with unlabeled nodes to improve generalization. However, most existing PRL approaches suffer from overfitting in an end-to-end setting or cannot encode task-specific information when used as unsupervised pre-training (i.e., two-stage learning). Meanwhile, LDR strategies often introduce redundant and invalid data perturbations that can slow down and mislead the training. To address all these issues, we propose a general framework SemiGraL for semi-supervised learning on graphs, which bridges and facilitates both PRL and LDR in a single shot. By extending a contrastive learning architecture to the semi-supervised setting, we first develop a semi-supervised contrastive representation learning process with virtual adversarial augmentation to map input nodes into a label-preserving representation space while avoiding overfitting. We then introduce a multiview consistency classification process with well-constrained perturbations to achieve adversarially robust classification. Extensive experiments on seven semi-supervised node classification benchmark datasets show that SemiGraL outperforms various baselines while enjoying strong generalization and robustness performance.

Abstracts:Outlier detection methods based on distance measure have been used in numerous applications due to their effectiveness and interpretability. However, distances among instances heavily depend on the feature space in which they reside. For an outlier, distances from it to the normal instances may be extremely close in one feature space, failing to separate them from each other, while this situation is reversed in another space. Meanwhile, the distance measure is sensitive to a few “marginal instances” (i.e., normal instances located very close to outliers in the feature space) during the estimation of whether a test instance is an outlier or not. In this article, we propose a robust multi-kernel nearest neighborhood (RMKN) method for outlier detection. Specifically, in the training phase, we only consider normal instances and transform them into a Polynomial kernel function weighted digraph to capture their geometric relationships in the original feature space. Then, we develop an objective function based on the weighted digraph to find a latent feature space via multi-kernel learning such that distances among normal instances in this latent feature space are as close as possible while preserving their original distributions. In the detecting phase, we design an outlying score based on the two-stage multi-kernel $k$k-nearest nearest neighbors to detect outliers. Extensive experiments with ten datasets show that RMKN is effective and robust.

Abstracts:Anchor-based multi-view graph clustering has recently gained popularity as an effective approach for clustering data with multiple views. However, existing methods have limitations in terms of handling inconsistent information and noise across views, resulting in an unreliable consensus representation. In addition, post-processing is needed to obtain final results after anchor graph construction, which negatively affects clustering performance. In this article, we propose a Robust and Consistent Anchor Graph Learning method (RCAGL) for multi-view clustering to address these challenges. RCAGL constructs a consistent anchor graph that captures inter-view commonality and filters out view-specific noise by learning a consistent part and a view-specific part simultaneously. A $k$k-connectivity constraint is imposed on the consistent anchor graph, leading to a clear graph structure and direct generation of cluster labels without additional post-processing. Experimental results on several benchmark datasets demonstrate the superiority of RCAGL in terms of clustering accuracy, scalability to large-scale data, and robustness to view-specific noise, outperforming advanced multi-view clustering methods.

Abstracts:Imbalanced learning is a traditional yet hot research subarea in machine learning. There are a huge number of imbalanced learning methods proposed in previous literature. This study focuses on one of the most popular imbalanced learning strategies, namely, sample reweighting. The key issue is how to calculate the weights of samples in training. While most studies have relied on intuitive theoretical or heuristic inspirations, few studies have attempted to establish a comprehensive theoretical path for weight calculation. A recent study utilizes the effective number theory for random covering to construct a theoretical weighting framework. In this study, we conduct a deep analysis to theoretically reveal the defects in the existing effective number-based weighting theory. An enhanced effective number theory is established in which data scatter and covering offset among different categories are involved. Subsequently, a new weight calculation manner is proposed based on our new theory, yielding a new loss, namely, NENum loss. In this loss, weights are sample-wise instead of category-wise used in the existing effective number-based weighting. Furthermore, another novel loss that combines weighting and logit perturbation is designed inspired the limitations of the NENum loss. Meta learning is employed to optimize the concrete calculation based on sample-wise training dynamics. We conduct extensive experiments on benchmark imbalanced and standard data corpora. Results validate the reasonableness of our enhanced theory and the effectiveness of the proposed methodology.