This repository contains the python implementation for paper "Second-Order Unsupervised Feature Selection via Knowledge Contrastive Distillation".
Unsupervised feature selection aims to select a subset from the original features that are most useful for the downstream tasks without external guidance information. While most unsupervised feature selection methods focus on ranking features based on the intrinsic properties of data, they do not pay much attention to the relationships between features, which often leads to redundancy among the selected features. In this paper, we propose a two-stage Second-Order unsupervised Feature selection via knowledge contrastive disTillation (SOFT) model that incorporates the second-order covariance matrix with the first-order data matrix for unsupervised feature selection. In the first stage, we learn a sparse attention matrix that can represent second-order relations between features. In the second stage, we build a relational graph based on the learned attention matrix and adopt graph segmentation for feature selection. Experimental results on 12 public datasets demonstrate the effectiveness of our proposed method.
- faiss
- h5py
- networkx
- numpy
- nxmetis
- pandas
- pytorch
- scipy
- skfeature
clustering.py: clustering method to get pseudo labelsevaluate.py: the second stage of SOFT to select featuresmodel.py: pytorch implementation of SOFT modelselector.py: the first stage of SOFT to learn second-order feature relationsutil.py: support functions
In the first stage, SOFT learns the second-order feature relation matrix by running:
python selector.pyIn the second stage, SOFT selects features and is evaluated by running:
python evaluate.py