Tech issue with Wallet doing 1500+ transactions per day

ec1warc1

ec1warc1

Active member
Hello everyone! As you may know, I run a website, dash.red, that is a fun way for people to win duffs. We are currently doing about 1600 transactions ever 24 hours.

We have a technical issue.

Sometimes, the wallet will be doing its thing - which is fairly constant transactions paying 30,000 duffs - when suddenly, there are very little or no funds available. We go from 30 million duffs to 16 thousand duffs in an instant. Now, there are not enough funds to pay anyone.

Scary!

The solution is to stop the dashd, and start the dashd again. There, all the funds are back in place. It takes about 30 seconds.

The problem with this solution is that I cannot be checking the wallet every 5 minutes. I cannot be restarting it every 5 minutes either.

The more transactions per day that we do, the more frequent this is going to occur. I assume it has something to do with one transaction jumping on top of another, but that may not be the case.

Two questions:
  1. Is there a dash-cli command to get the wallet to quickly review its funds again without restarting dashd?
  2. Is there anyone from Core team that wants to look at our logs to see if there is a problem with a potential solution or perhaps even a bug?
Thanks,

ec1warc1
 
ec1warc1 said:
...
Two questions:
  1. Is there a dash-cli command to get the wallet to quickly review its funds again without restarting dashd?
  2. Is there anyone from Core team that wants to look at our logs to see if there is a problem with a potential solution or perhaps even a bug?
...
Click to expand...
1. Don't think so, but it's hard to tell because it's not clear what could cause the issue
2. Absolutely. Please send me debug.log via email.
 
I just had this happen again - twice in about 30 minutes. Could this be an example of the problem:
2018-01-11 17:47:23 keypool keep 9173
2018-01-11 17:47:23 AddToWallet 9b07d9b096580861e58ae70504df8e42a3da3f021079cda5369a30f8bc65b9ec new
2018-01-11 17:47:23 AddToWallet 9b07d9b096580861e58ae70504df8e42a3da3f021079cda5369a30f8bc65b9ec
2018-01-11 17:47:23 Relaying wtx 9b07d9b096580861e58ae70504df8e42a3da3f021079cda5369a30f8bc65b9ec
2018-01-11 17:47:32 keypool added key 10173, size=1000, internal=0
2018-01-11 17:47:32 init message: Loading wallet... (1016.28 %)
2018-01-11 17:47:32 keypool reserve 9174
2018-01-11 17:47:32 CommitTransaction:
CTransaction(hash=f1b72c4f5e, ver=1, vin.size=1, vout.size=2, nLockTime=802501)
CTxIn(COutPoint(9b07d9b096580861e58ae70504df8e42a3da3f021079cda5369a30f8bc65b9ec, 1), scriptSig=47304402206b3bbff31f14de, nSequence=4294967294)
CTxOut(nValue=0.00030000, scriptPubKey=76a9148961b2aa63739d6161f981e6)
CTxOut(nValue=0.10338955, scriptPubKey=76a914cb918afe6e7d991eed7f1af2)
2018-01-11 17:47:32 keypool keep 9174
2018-01-11 17:47:32 AddToWallet f1b72c4f5e53d36b46cdbfd58001520bced9bc36584db5e285b5a6ab477c7d0f new
2018-01-11 17:47:32 CommitTransaction(): Error: Transaction not valid
2018-01-11 17:47:54 CMasternodeMan::CheckAndRemove
 
It just happened again, third time today. It seems that the error is in the logs with this text:
"2018-01-11 18:12:08 CommitTransaction(): Error: Transaction not valid"
I was running version dashcore-0.12.2.1 and just updated it to dashcore-0.12.2.2.

It would be great if the update of the software version fixed the problem. I will let everyone know here in this thread.

Hello demo! Nice to hear from you. I use rpc connection written in php to connect from the application server across the network to the dashd.
 
ec1warc1 said:
It just happened again, third time today. It seems that the error is in the logs with this text:
"2018-01-11 18:12:08 CommitTransaction(): Error: Transaction not valid"
I was running version dashcore-0.12.2.1 and just updated it to dashcore-0.12.2.2.

It would be great if the update of the software version fixed the problem. I will let everyone know here in this thread.

Hello demo! Nice to hear from you. I use rpc connection written in php to connect from the application server across the network to the dashd.
Click to expand...

What kind of application server? You may have to reduce the throughput of your application server towards dashd.

Read below:

https://bitcoin.stackexchange.com/q...transaction-throughput-of-the-bitcoin-network

If people are right in the above stackexchange thread, bitcoind allows 3-7 transactions per second. Dashd use the same database than bitcoin uses (I think it is berkleydb or leveldb), but the blocksize of dash is doubled, so thats the reason the problem occurs.

@UdjinM6 may inform us what is the maximum throughput of dashd.
 
Last edited:
When the error occurs, I don't think we are not getting contention from other transactions. Here is one way to view and compare the last two transactions which happened 8 seconds apart and the second one with the error:
ec1warc1@server:~/.dashcore $ grep "2018-01-11 18:12:00" debug.log
2018-01-11 18:12:00 keypool added key 10197, size=1000, internal=0
2018-01-11 18:12:00 init message: Loading wallet... (1018.68 %)
2018-01-11 18:12:00 keypool reserve 9198
2018-01-11 18:12:00 CommitTransaction:
2018-01-11 18:12:00 keypool keep 9198
2018-01-11 18:12:00 AddToWallet 5ac310924802a71e84b51717bef48af1a3eec2fccc86ddf31099756ab8a6f04d new
2018-01-11 18:12:00 AddToWallet 5ac310924802a71e84b51717bef48af1a3eec2fccc86ddf31099756ab8a6f04d
2018-01-11 18:12:00 Relaying wtx 5ac310924802a71e84b51717bef48af1a3eec2fccc86ddf31099756ab8a6f04d
ec1warc1@server:~/.dashcore $ grep "2018-01-11 18:12:08" debug.log
2018-01-11 18:12:08 keypool added key 10198, size=1000, internal=0
2018-01-11 18:12:08 init message: Loading wallet... (1018.78 %)
2018-01-11 18:12:08 keypool reserve 9199
2018-01-11 18:12:08 CommitTransaction:
2018-01-11 18:12:08 keypool keep 9199
2018-01-11 18:12:08 AddToWallet e91caf5031419c45a662e66badb7d62989b45795649079ed3ff058ed70895369 new
2018-01-11 18:12:08 CommitTransaction(): Error: Transaction not valid
ec1warc1@server:~/.dashcore $
 
Regarding tuning the app server and other servers. I have the three servers isolated, talking to each other over the network. The app is LAMP for wordpress, without the DB. Then, there are the DB server and the wallet server. Each on its own OS.
 
In a private conversation, I got this information from UdjinM6, which after some study looks promising. I am going to post an image of his answer here for others to read:
upload_2018-1-11_18-3-38.png
 
It looks like a good start of solving this is to simply send duffs to different addresses in the wallet, and not all duffs to the same address as I have done in the past.
Perhaps also to include the parameter spendzeroconfchange=0 to the config file.

Do I have that right @UdjinM6? Thanks sooo much!!!!
 
@ec1warc1 Most likely unrelated to your actual problem, but from the log snippets I see that your keypool has ran out of pooled keys since quite some time. That means, if you don't do VERY regular backups atm, you'll loose some (or all?) of your Dash in case you loose your hot wallet. Old backups will be useless as they won't contain newly generated keys.

I'd suggest to at least call "keypoolrefill 20000" right now. Please do a backup before and after you do this. Only the newest backup after that call will help you in future restores.
 
ec1warc1 said:
It looks like a good start of solving this is to simply send duffs to different addresses in the wallet, and not all duffs to the same address as I have done in the past.
Perhaps also to include the parameter spendzeroconfchange=0 to the config file.

Do I have that right @UdjinM6? Thanks sooo much!!!!
Click to expand...

Even better run two or more dashd and make your application server do a round-robin to them.
 
Just as an update... it has been a week. I added 4 addresses to the wallet and started distributing the duffs among the addresses. Today, we had our first error in 7 days, so a big improvement, just not perfect yet. Here is today's error:
2018-01-19 23:00:37 CommitTransaction(): Error: Transaction not valid
 
hi guys hi @ec1warc1

we are trying to process dash transactions as well ,we got about 1000-2000 per day, let me know how you have achieved , i have tried very many things but failed

if you have a detailed process or something i would really appreciate that.
 
Dashmaximalist said:
hi guys hi @ec1warc1

we are trying to process dash transactions as well ,we got about 1000-2000 per day, let me know how you have achieved , i have tried very many things but failed

if you have a detailed process or something i would really appreciate that.
Click to expand...

Step 1. Buy three linux dedicated servers.
ec1warc1 said:
Regarding tuning the app server and other servers. I have the three servers isolated, talking to each other over the network. The app is LAMP for wordpress, without the DB. Then, there are the DB server and the wallet server. Each on its own OS.
Click to expand...
 
Last edited:
Dashmaximalist said:
hi guys hi @ec1warc1

we are trying to process dash transactions as well ,we got about 1000-2000 per day, let me know how you have achieved , i have tried very many things but failed

if you have a detailed process or something i would really appreciate that.
Click to expand...

Did you verify that you do not suffer from the same problem as reported by ec1warc1? As UdjinM6 explained, the problem is most likely that too many transactions are sitting unconfirmed in the mempool while you try to send new transactions which spend the unconfirmed transaction outputs. The mempool is able to store many transactions, but seems to have limits when it comes to chained unconfirmed transactions. I did not check that specific code, but it looks like this might be intended as spam protection.

To solve this, you have to split your balances into multiple outputs, so that your wallet has more confirmed outputs to choose from when building new transactions. To achieve this, you can use the "split outputs/utxos" (I don't have a running wallet at hand atm and don't exactly know how the text is) checkbox and enter the number of outputs in the right field of it. Try to send some Dash to yourself with this checkbox set and a reasonable amount of splitted outputs. The amount of Dash and number of outputs depends on how large the outgoing transactions and the amount of them you expect. You should verify the number of available UTXOs from time to time and repeat this process.

Dash News Network said:
So Dash is marketed as being ready to scale, yet it utterly fails when asked to process a trivial 1000-2000 transactions per day?

That sounds like false advertising to me. Bitcoin Core has no trouble at all with doing that many transactions in an hour, much less in an entire day.

What is so wrong with Dash Core that they managed to break the code they copied from Bitcoin Core so badly that it can't even do less than 100 transactions per hour?
Click to expand...

You are making a bad comparison here. The numbers you always hear about Bitcoin being able to process is GLOBALLY and not PER NODE. If you follow the same spending behavior as described by UdjinM6, you'll very likely hit the same limits, maybe even worse due to longer confirmation times.
 
codablock said:
Did you verify that you do not suffer from the same problem as reported by ec1warc1? As UdjinM6 explained, the problem is most likely that too many transactions are sitting unconfirmed in the mempool while you try to send new transactions which spend the unconfirmed transaction outputs. The mempool is able to store many transactions, but seems to have limits when it comes to chained unconfirmed transactions. I did not check that specific code, but it looks like this might be intended as spam protection.
Click to expand...

Can this be solved by compiling the code and fixing the spam protection limits?
 
You must log in or register to reply here.
Back
Top