ETH & ETC uncle blokcs are orphaned. (Fix inside)

[ucyildirim]

Description

Uncle blocks on Ethereum family coins are getting orphaned.

Steps to Reproduce

0. Set EthereumConstants.MinConfimations = 100
1. Start Pool
2. Mine ETH or ETC and find uncle block,
3. Wait for the confirmation progress.

Expected behavior

That uncle block should be clasified as uncle.

Actual behavior

That block clasified as ORPHAN. Because of the line : https://github.com/coinfoundry/miningcore/blob/df0f06f44eddc8a8b7b5f6aff5f15c667130f079/src/Miningcore/Blockchain/Ethereum/EthereumPayoutHandler.cs#L240

Reproduces how often

Always.

Possible Fix

Replacing this line to if(block.Status == BlockStatus.Pending && **block.ConfirmationProgress > 1**) should solve the problem.

I think the main problem is, the classifier rouitine cannot reach to the EthereumConstants.MinConfimations due to ConfirmationProgress > 0.75 statement. When the ConfirmationProgress hits 0.75, classifies the block as an orphan. I think it should be orphan if ConfirmationProgress > 1

[oliverw]

@uurcm16 I'm currently not working on the ETH family since CF no longer runs them. Feel free to submit a PR.

[ucyildirim]

@uurcm16 I'm currently not working on the ETH family since CF no longer runs them. Feel free to submit a PR.

Ok, I'll submit a PR for that. Here is another issue with the SQL queries,

https://github.com/coinfoundry/miningcore/blob/df0f06f44eddc8a8b7b5f6aff5f15c667130f079/src/Miningcore/Persistence/Postgres/Repositories/StatsRepository.cs#L135

This line counts every single share to show non-important Pending Shares data. And that query takes 300+ms to complete on 2 x Xeon 36 core dedicated server.

I just removed that line for my pool because I don't need that. But I just wanted to let you know to see if there is anything that you could do to improve its performance.

And I know that MiningCore deletes shares after a while but that query still might be a problem on large pools

[oliverw]

This line counts every single share to show non-important Pending Shares data. And that query takes 300+ms to complete on 2 x Xeon 36 core dedicated server.

You're running a Multipool right?

[ucyildirim]

This line counts every single share to show non-important Pending Shares data. And that query takes 300+ms to complete on 2 x Xeon 36 core dedicated server.

You're running a Multipool right?

Yes I do.

[oliverw]

@uurcm16 This makes quite a difference: https://github.com/coinfoundry/miningcore#advanced-postgresql-database-setup

P.S.: And don't tell me upgrading a production pool to this is like juggling torches. Been there, done that 😁

[ucyildirim]

And don't tell me upgrading a production pool to this is like juggling torches

Ahaha, I did that once too. It is a real problem. But will do it anyways :D

I did see the upgrading notes and currently testing it on a local pool :). I'm not familiar with the partitioning system of the PostgreSQL. Here is the question, you are using Raw Insert mode to write the shares to the table. Does partitioning improve that too?

And I dont want to bother you / anyone with multiple issues but there are a few things that I've fixed.

1. When I run API server and pool as separated instances, API instance updates pool stats (Even if I use "EnableInternalStratum" : false).
2. Dag file generation is triggered by SHARES which miners submits. If no miner exists on a running pool, new epoch's dag file is not generated, and does not allow miner to connect or mine.
3. System cannot detect network and chain on ETC pool.

Do you want me to submit a PR for them?

[oliverw]

And don't tell me upgrading a production pool to this is like juggling torches

Ahaha, I did that once too. It is a real problem. But will do it anyways :D

I did see the upgrading notes and currently testing it on a local pool :). I'm not familiar with the partitioning system of the PostgreSQL. Here is the question, you are using Raw Insert mode to write the shares to the table. Does partitioning improve that too?

Yes, insert performance is also boosted by partitioning.

And I dont want to bother you / anyone with multiple issues but there are a few things that I've fixed.

1. When I run API server and pool as separated instances, API instance updates pool stats (Even if I use "EnableInternalStratum" : false).
2. Dag file generation is triggered by SHARES which miners submits. If no miner exists on a running pool, new epoch's dag file is not generated, and does not allow miner to connect or mine.
3. System cannot detect network and chain on ETC pool.

Yeah that's inconvenient.

Do you want me to submit a PR for them?

Sure

[ucyildirim]

And don't tell me upgrading a production pool to this is like juggling torches

Ahaha, I did that once too. It is a real problem. But will do it anyways :D
I did see the upgrading notes and currently testing it on a local pool :). I'm not familiar with the partitioning system of the PostgreSQL. Here is the question, you are using Raw Insert mode to write the shares to the table. Does partitioning improve that too?

Yes, insert performance is also boosted by partitioning.

And I dont want to bother you / anyone with multiple issues but there are a few things that I've fixed.

1. When I run API server and pool as separated instances, API instance updates pool stats (Even if I use "EnableInternalStratum" : false).
2. Dag file generation is triggered by SHARES which miners submits. If no miner exists on a running pool, new epoch's dag file is not generated, and does not allow miner to connect or mine.
3. System cannot detect network and chain on ETC pool.

Yeah that's inconvenient.

Do you want me to submit a PR for them?

Sure

Ok, I'll submit PR asap.

Here is the question; I think you are using dag files to process shares faster. Can't we get rid of the Dag Files like open-ethereum-pool?
What is the difference between instantly calculating the share and using a dag file? Is it too slow to calculate instantly?

[oliverw]

open-ethereum-pool utilizes the DAG as well. I don't think there's a way around this. If you dig into open-ethereum-pool's dependencies you will discover this package which handles the DAG stuff: https://github.com/ethereum/ethash

[ucyildirim]

open-ethereum-pool utilizes the DAG as well. I don't think there's a way around this. If you dig into open-ethereum-pool's dependencies you will discover this package which handles the DAG stuff: https://github.com/ethereum/ethash

Yes, i did see that the OEP using that package.
Here is the line that I mentioned.
https://github.com/sammy007/open-ethereum-pool/blob/bcfc0eb0e683993ad0c51bacad8d4b354e769393/proxy/miner.go#L44

And I actually didn't notice anything about the dag file generation while pool is running.

[oliverw]

Because you simply can't trust people these days. Not validating shares pool-side means everyone can submit junk shares and get paid for it on the backs of honest miners.

And I actually didn't notice anything about the dag file generation while pool is running.

Check the home directory of the user under which OEP runs. Found it?

[saltyminer]

@uurcm16 have you done any work with this? I would like to help.

[ucyildirim]

@uurcm16 have you done any work with this? I would like to help.

I'll submit PR for all of the problems above.
I was too busy last 3 weeks.

[ucyildirim]

@uurcm16 have you done any work with this? I would like to help.

I might need help with this :D => #656

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[yq0810]

New pull request
#1574