TAG-CF: Test-time Aggregation for CF

Source code for the paper How Does Message Passing Improve Collaborative Filtering? accepted at NeurIPS 2024.

by Mingxuan Ju, William Shiao, Zhichun Guo, Fanny Ye, Yozen Liu, Neil Shah and Tong Zhao.

The paper proposes TAG-CF which is a test-time aggregation framework that can be utilized as a plug-and-play module to enhance the performance of matrix factorization models.

1. Installation

Please install all dependencies using the command:

conda create --name <env> --file requirements.txt

2. Download data

Please download the data following the instruction.

3. Run experiments

This is one example of reproducing results for Movielens-1M with BPR. Please use other provided config files to reproduce results for other datasets.

# Train a basic matrix factorization model
make run_pipeline MODEL_YAML_PATH=config/MF.yaml DATA_YAML_PATH=config/bpr_ml.yaml

# Train a LightGCN model.
make run_pipeline MODEL_YAML_PATH=config/LGCN.yaml DATA_YAML_PATH=config/bpr_ml.yaml

# Conduct test-time aggregation on matrix factorization model.
make run_pipeline MODEL_YAML_PATH=config/TAGCF_bpr_ml.yaml  DATA_YAML_PATH=config/bpr_ml.yaml

4. Results

This is a list containing performance of MF trained by BPR and DirectAU, along with the relative performance improvement brought by TAG-CF. We provide hyper-parameter configs in the config folder. To play around with m and n in TAG-CF, feel free to accordingly change individual config files. To conduct grid-search over them, simply remove them from the config file.

	DirectAU	+TAGCF	Impr. (%)	BPR	+TAGCF	Impr. (%)
			Amazon-Book
Recall@10	0.0619	0.0630	1.83%	0.0396	0.0454	14.55%
Recall@20	0.0959	0.0972	1.32%	0.0652	0.0737	13.04%
NDCG@10	0.0527	0.0536	1.60%	0.0330	0.0384	16.47%
NDCG@20	0.0641	0.0650	1.54%	0.0416	0.0479	15.06%
			Yelp-2018
Recall@10	0.0618	0.0633	2.44%	0.0456	0.0475	4.25%
Recall@20	0.0997	0.1018	2.09%	0.0761	0.0793	4.15%
NDCG@10	0.0517	0.0532	2.80%	0.0385	0.0403	4.68%
NDCG@20	0.0647	0.0663	2.56%	0.0489	0.0513	4.71%
			Gowalla
Recall@10	0.1045	0.1072	1.72%	0.0869	0.0914	5.15%
Recall@20	0.1528	0.1566	1.71%	0.1272	0.1333	4.73%
NDCG@10	0.0825	0.0839	2.66%	0.0690	0.0719	4.16%
NDCG@20	0.0976	0.0993	2.50%	0.0816	0.0850	4.17%
			MovieLens-1M
Recall@10	0.1223	0.1602	31.00%	0.1276	0.1319	3.35%
Recall@20	0.1899	0.2452	29.14%	0.2179	0.2187	0.35%
NDCG@10	0.1521	0.2085	37.03%	0.1528	0.1571	2.82%
NDCG@20	0.1667	0.2233	33.93%	0.1800	0.1821	1.15%

5. Reference

If you find this repo and our work useful to you, please kindly cite us using:

@article{ju2024does,
  title={How Does Message Passing Improve Collaborative Filtering?},
  author={Ju, Mingxuan and Shiao, William and Guo, Zhichun and Ye, Yanfang and Liu, Yozen and Shah, Neil and Zhao, Tong},
  journal={Advances in neural information processing systems},
  year={2024}
}

6. Contact

Please contact mju@snap.com for any questions.