Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider calling glm.fit() instead of glm() #569

Open
mbojan opened this issue Jul 4, 2024 · 10 comments
Open

Consider calling glm.fit() instead of glm() #569

mbojan opened this issue Jul 4, 2024 · 10 comments

Comments

@mbojan
Copy link
Member

mbojan commented Jul 4, 2024

In

ergm/R/ergm.mple.R

Lines 100 to 101 in 1f4401e

glm.result <- quietly(function() glm(pl$zy ~ .-1 + offset(pl$foffset),
data=data.frame(pl$xmat), weights=pl$wend, family=family))()

consider calling glm.fit() directly rather than glm(). Experiments with biggish data show that it might cut the computing time by half.

@krivit
Copy link
Member

krivit commented Aug 18, 2024

@mbojan , can you test to see if this works?

@mbojan mbojan self-assigned this Aug 29, 2024
@krivit
Copy link
Member

krivit commented Sep 23, 2024

@mbojan , I'll be submitting an update to ergm in the next few days. If you want this to go in, let me know ASAP.

@mbojan
Copy link
Member Author

mbojan commented Sep 23, 2024

Oh dear, I have a week of workshops, including ergms. Can we release next week? The principle answer is yes, but haven't tested yet.

@krivit
Copy link
Member

krivit commented Sep 28, 2024

Oh dear, I have a week of workshops, including ergms. Can we release next week? The principle answer is yes, but haven't tested yet.

OK, can you get it done in the next day or two?

@krivit
Copy link
Member

krivit commented Oct 2, 2024

@mbojan ?

@krivit
Copy link
Member

krivit commented Oct 3, 2024

@AdrienLeGuillou , you often fit MPLE to large networks, right? Can you by any chance test this?

@AdrienLeGuillou
Copy link
Member

I just ran a quick test on a smaller 10k nodes network using this branch. It worked fine. I can't tell if it was faster or not as I usually work with "Stochastic-Approximation" on these smaller local tests.
I can try on the HPC with our 3 - 100k nodes networks and compare the time it takes.

@AdrienLeGuillou
Copy link
Member

I just realized that ergm.mple is called whatever the main.method we use.
Therefore I can confirm that it works perfectly on our 10k nodes networks.
It takes a very similar amount of time to fit the networks with both version as the MPLE step is not the longest part anyways.

@AdrienLeGuillou
Copy link
Member

I confirm it also works on the 100k nodes network.
It was actually longer with glm.fit. But the difference was on the number of MCMLE iterations.

@mbojan
Copy link
Member Author

mbojan commented Oct 3, 2024

Thanks @AdrienLeGuillou . @krivit don't merge, leave as is. I need to dig out the script where I think I noticed the difference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants