need help in reproducing the results in your paper

[ProgramMonkey]

Hi dingjingtao,

Thank you for sharing your implementation for your paper which is very inspiring.

However, I failed to reproduce the recommendation accuracies presented in the paper and even after tuning the parameters based on the paper.

Taking the dataset Tmall given in this project as an example, when using the default settings in main_MF for VALS, the HR and NDCG are only 0.0083 and 0.0020, respectively, by Iteration 20, and after tuning the parameters, HR and NDCG can still hardly exceed 0.035 by Iteration 20. These results are far lower than those presented in Figure 3 in the paper, where HR and NDCG exceed 0.06 and 0.015, respectively, in the first several iterations.

Could you please share more details about the parameters for achieving the highest recommendation accuracies? or give me some hints to achieve the results presented in the paper?

I list the default and my tunned parameters in the file main_MF below,

the default:
String dataset_name = "ijcai18-vals-master\data\tmall_all\Tmall_purchase";
String sidedataset_name = "ijcai18-vals-master\data\tmall_all\Tmall_view";
String method = "vieweALS";
double w0 = 0.5; // c0 in the paper, given
double w1 = 1; //s0 in the paper, not sure
double w2 = 1; //seems not used by any algorithm
double r1 = 1;
boolean showProgress = false;
boolean showLoss = true;
int factors = 64;
int maxIter = 500;
double reg = 0.01;
double alpha = 0.75;
double beta = 0.2;
double ratio = 0;
double gamma1 = 0;
double gamma2 = 0;

my tunned :
String dataset_name = "ijcai18-vals-master\data\tmall_all\Tmall_purchase";
String sidedataset_name = "ijcai18-vals-master\data\tmall_all\Tmall_view";
String method = "vieweALS";
double w0 = 0.5; // c0 in the paper, given
double w1 = 800; //s0 in the paper, not sure
double w2 = 1; //seems not used by any algorithm
double r1 = 1;
boolean showProgress = false;
boolean showLoss = true;
int factors = 32; // all baselines are set to 32
int maxIter = 500; // is set by default and does not matter
double reg = 0.001;
double alpha = 0.5; //not sure
double beta = 0.5; //not sure, seems that when alpha = 2 and beta = 0.5, the results is good
double ratio = 0;
double gamma1 = 3.5; // given explicitly
double gamma2 = 3.5; // given explicitly

Thank you for your consideration and time!

[dingjingtao]

Hi dingjingtao,

Thank you for sharing your implementation for your paper which is very inspiring.

However, I failed to reproduce the recommendation accuracies presented in the paper and even after tuning the parameters based on the paper.

Taking the dataset Tmall given in this project as an example, when using the default settings in main_MF for VALS, the HR and NDCG are only 0.0083 and 0.0020, respectively, by Iteration 20, and after tuning the parameters, HR and NDCG can still hardly exceed 0.035 by Iteration 20. These results are far lower than those presented in Figure 3 in the paper, where HR and NDCG exceed 0.06 and 0.015, respectively, in the first several iterations.

Could you please share more details about the parameters for achieving the highest recommendation accuracies? or give me some hints to achieve the results presented in the paper?

I list the default and my tunned parameters in the file main_MF below,

the default:
String dataset_name = "ijcai18-vals-master\data\tmall_all\Tmall_purchase";
String sidedataset_name = "ijcai18-vals-master\data\tmall_all\Tmall_view";
String method = "vieweALS";
double w0 = 0.5; // c0 in the paper, given
double w1 = 1; //s0 in the paper, not sure
double w2 = 1; //seems not used by any algorithm
double r1 = 1;
boolean showProgress = false;
boolean showLoss = true;
int factors = 64;
int maxIter = 500;
double reg = 0.01;
double alpha = 0.75;
double beta = 0.2;
double ratio = 0;
double gamma1 = 0;
double gamma2 = 0;

my tunned :
String dataset_name = "ijcai18-vals-master\data\tmall_all\Tmall_purchase";
String sidedataset_name = "ijcai18-vals-master\data\tmall_all\Tmall_view";
String method = "vieweALS";
double w0 = 0.5; // c0 in the paper, given
double w1 = 800; //s0 in the paper, not sure
double w2 = 1; //seems not used by any algorithm
double r1 = 1;
boolean showProgress = false;
boolean showLoss = true;
int factors = 32; // all baselines are set to 32
int maxIter = 500; // is set by default and does not matter
double reg = 0.001;
double alpha = 0.5; //not sure
double beta = 0.5; //not sure, seems that when alpha = 2 and beta = 0.5, the results is good
double ratio = 0;
double gamma1 = 3.5; // given explicitly
double gamma2 = 3.5; // given explicitly

Thank you for your consideration and time!

Please follow the Table3+last para of Sec.5.2 in the paper
run jar as
"java -jar <name_of_jar>.jar main_MF <path_of_purchase_file> vieweALS 800 False True 32 200 0.01 0 <path_of_view_file> 0.5 0 3.5 3.5"

[ProgramMonkey]

Hi dingjingtao,
Thank you for sharing your implementation for your paper which is very inspiring.
However, I failed to reproduce the recommendation accuracies presented in the paper and even after tuning the parameters based on the paper.
Taking the dataset Tmall given in this project as an example, when using the default settings in main_MF for VALS, the HR and NDCG are only 0.0083 and 0.0020, respectively, by Iteration 20, and after tuning the parameters, HR and NDCG can still hardly exceed 0.035 by Iteration 20. These results are far lower than those presented in Figure 3 in the paper, where HR and NDCG exceed 0.06 and 0.015, respectively, in the first several iterations.
Could you please share more details about the parameters for achieving the highest recommendation accuracies? or give me some hints to achieve the results presented in the paper?
I list the default and my tunned parameters in the file main_MF below,
the default:
String dataset_name = "ijcai18-vals-master\data\tmall_all\Tmall_purchase";
String sidedataset_name = "ijcai18-vals-master\data\tmall_all\Tmall_view";
String method = "vieweALS";
double w0 = 0.5; // c0 in the paper, given
double w1 = 1; //s0 in the paper, not sure
double w2 = 1; //seems not used by any algorithm
double r1 = 1;
boolean showProgress = false;
boolean showLoss = true;
int factors = 64;
int maxIter = 500;
double reg = 0.01;
double alpha = 0.75;
double beta = 0.2;
double ratio = 0;
double gamma1 = 0;
double gamma2 = 0;
my tunned :
String dataset_name = "ijcai18-vals-master\data\tmall_all\Tmall_purchase";
String sidedataset_name = "ijcai18-vals-master\data\tmall_all\Tmall_view";
String method = "vieweALS";
double w0 = 0.5; // c0 in the paper, given
double w1 = 800; //s0 in the paper, not sure
double w2 = 1; //seems not used by any algorithm
double r1 = 1;
boolean showProgress = false;
boolean showLoss = true;
int factors = 32; // all baselines are set to 32
int maxIter = 500; // is set by default and does not matter
double reg = 0.001;
double alpha = 0.5; //not sure
double beta = 0.5; //not sure, seems that when alpha = 2 and beta = 0.5, the results is good
double ratio = 0;
double gamma1 = 3.5; // given explicitly
double gamma2 = 3.5; // given explicitly
Thank you for your consideration and time!

Please follow the Table3+last para of Sec.5.2 in the paper
run jar as
"java -jar <name_of_jar>.jar main_MF <path_of_purchase_file> vieweALS 800 False True 32 200 0.01 0 <path_of_view_file> 0.5 0 3.5 3.5"

[ProgramMonkey]

Thank you very much for your prompt reply. It works. It seems that I confused w0 and w1 previously.