Dashmate (discussion)

@pshenmic

I tested .deb installation of Dashmate v0.25.15 & v0.25.16-rc.2 on my OS arch (Ubuntu 20.04.6 LTS / GNU/Linux 5.4.0-166-generic x86_64)

wget https://github.com/dashpay/platform/releases/download/v0.25.15/dashmate_0.25.15-1_amd64.deb
sudo dpkg -i dashmate_0.25.15-1_amd64.deb
dashmate update

dashpay/dashmate-helper:0.25.13 -> up to date
dashpay/dashd:20.0.1 -> up to date

wget https://github.com/dashpay/platform...6-rc.2/dashmate_0.25.16.56ce20547-1_amd64.deb
sudo dpkg -i dashmate_0.25.16.56ce20547-1_amd64.deb
dashmate update

dashpay/dashmate-helper:0.25.13 -> up to date
dashpay/dashd:20.0.1 -> up to date

So this downloaded and installed v0.25.15 & 0.25.16-rc.2 through .deb methode, but still shows dashmate version 0.25.13 as current version, after a 'dashmate update' command. No problem with any commands and node is running fine.

Is there a specific command available to verify Dashmate current version, besides using 'dashmate update' ?

To summarize my problems :

Through npm download and installation of Dashmate (v0.25.15 & v0.25.16-rc.x), i am getting errors and problems with dashmate commands
Through .deb download and installation of Dashmate (v0.25.15 & v0.25.16-rc.x), i am getting either an incorrect Dashmate version shown through 'dashmate update' command or Dashmate actually got stuck on v0.25.13
 
Last edited:
Bridgewater said:
Will these releases start being signed at some point? With your keys or pastas, etc?
Click to expand...
I don't know, never thought about this yet. I think we should start building other components (Drive, Dapi, etc..) and upload them in the assets first. That's more question to the DCG though
Bridgewater said:
With dashmate core cli, does it not support any arguments or flags? I can't get it to work with more than one word commands. For example,
Click to expand...
Yes, you should wrap it in the "" and it should work, f.e:
ubuntu@ip-172-31-37-48:~$ dashmate core cli "masternodelist evo"
{
"6ce8545e25d4f03aba1527062d9583ae01827c65b234bd979aca5954c6ae3a59-32": {
"proTxHash": "8e11eb784883d3dc9d0d74a74633f067dc61c408dfdee49b8f93bb161f2916c0",
"address": "52.89.154.48:19999",
"payee": "yeRZBWYfeNE4yVUHV4ZLs83Ppn9aMRH57A",
"status": "POSE_BANNED",
"type": "Evo",
"platformNodeID": "6d57707c196e06487d326094c964f258f5b77c35",
"platformP2PPort": 36656,
"platformHTTPPort": 1443,
"pospenaltyscore": 516,
"consecutivePayments": 0,
"lastpaidtime": 1699548876,
"lastpaidblock": 909774,
"owneraddress": "yLMWYiFxCpPwim9YuosKAozkqsGf283XCa",
"votingaddress": "yLMWYiFxCpPwim9YuosKAozkqsGf283XCa",
"collateraladdress": "yigAyPgP394rpKR22XEDNQnoPCXUfXQYkz",
"pubkeyoperator": "8160877a911d8bb7d1e75e2320e98cc3233c1f6972cb642424bfcec7c182c56d2c0ebb59e45f788f4d5dbfa2ebff3e3a"
},
"6ce8545e25d4f03aba1527062d9583ae01827c65b234bd979aca5954c6ae3a59-5": {
"proTxHash": "f6496d3c0ac1ad94ff5e3aab2edcabc1c8ba3fbfea0ef3026e90ba7863990381",
"address": "52.13.250.182:19999",
.....
Click to expand...
I'm not sure that dash-cli arguments are fully supported, like if you want to override -rpcwallet, -rpchost and other, but commands should generally work

qwizzie said:
Through npm download and installation of Dashmate (v0.25.15 & v0.25.16-rc.x), i am getting errors and problems with dashmate commands
Through .deb download and installation of Dashmate (v0.25.15 & v0.25.16-rc.x), i am getting either an incorrect Dashmate version shown through 'dashmate update' command or Dashmate actually got stuck on v0.25.13
Click to expand...
I think NPM + .deb installations could conflict with each others, did you try installing your dashmate from NPM first?
`npm uninstall -g dashmate`
 
pshenmic said:
I think NPM + .deb installations could conflict with each others, did you try installing your dashmate from NPM first?
`npm uninstall -g dashmate`
Click to expand...
After uninstall of Dashmate, updating to Node.JS v20 and installing v0.25.16-rc.2 through .deb the same error :

CompileError: WebAssembly.Module(): Compiling function #332:"memchr::arch::wasm32::simd128: packedpair::Find..." failed: Wasm SIMD unsupported @+1951319

At least i got some hands-on experience with .deb installation. I also got hit with another PoSe score (after it just cleared a previous PoSe score), i guess my Evonode was a bit unlucky during the last stop and start of Dashmate during this test session. I think it could also be related to updating Node.JS to v20 once more (triggered by uninstall of Dashmate and reinstall of Dashmate v0.25.16-rc2 through .deb). So for now no more testing from me, the chance of getting PoSe scores during dashmate stop and start is a bit too high for my taste these days.

@pshenmic i am curious if you managed to reproduce my issue/error on my OS arch (Ubuntu 20.04.6 LTS) ?
 
Last edited:
No, unfortunately I could not reproduce your error. However I suspect one place in the code, I will confirm with the team.

I'm sorry that you got an PoSe ban. Reinstalling dashmate should not really affect the pose score, because everything is stored in Docker, but reinstalling node or dashmate does not make changes to it.
 
pshenmic said:
No, unfortunately I could not reproduce your error. However I suspect one place in the code, I will confirm with the team.

I'm sorry that you got an PoSe ban. Reinstalling dashmate should not really affect the pose score, because everything is stored in Docker, but reinstalling node or dashmate does not make changes to it.
Click to expand...
It was just a PoSe score (like the previous one). Should clear in a few days (-1 PoSe score per block).
Good luck with hunting down this bug.
 
pshenmic said:
Yes, you should wrap it in the "" and it should work, f.e:
Click to expand...
Ah, I was only wrapping the sub-commands/arguments in the " and didn't realize the whole thing needs to be wrapped. dashmate core cli protx "info protxhash" failed, but dashmate core cli "protx info protxhash" works!
 
@qwizzie
Regarding your issue with SIMD, what CPU do you have on your node?

The only change that happened between 0.25.13 and 0.25.15 is bumping minimum Rust version to 1.74.
The error message you are receiving I believe is happening on the command init phase when WasmDPP being loaded that is used in few places in the dashmate.

The reason could be that something was updated in the Rust compiler, or your CPU missing some instructions, that wasn't required before. Since we have pretty the same environment (OS distribution + prebuilts), I think the difference could be in the hardware.
 
@pshenmic

$lscpu

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 44 bits physical, 48 bits virtual
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 1
Core(s) per socket: 8
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel(R) Xeon(R) Gold 6136 CPU @ 3.00GHz
Stepping: 4
CPU MHz: 2992.972
BogoMIPS: 5985.94
Hypervisor vendor: Microsoft
Virtualization type: full
L1d cache: 256 KiB
L1i cache: 256 KiB
L2 cache: 8 MiB
L3 cache: 198 MiB
NUMA node0 CPU(s): 0-7

I think you should check this issue directly with Sam. If it is indeed Rust-compiler related, he should know how to solve this.
I think he was also the one that merged Dashmate v0.25.14 (failed mergers ?) and Dashmate v0.25.15

Knipsel.JPG


Maybe the removal of those unused dependencies of rust crates (#1578) has something to do with below issue with regards to Dashmate v0.25.15 & 0.25.16 (rc1 & rc2) on Mainnet ?

CompileError: WebAssembly.Module(): Compiling function #332:"memchr::arch::wasm32::simd128: packedpair::Find..." failed: Wasm SIMD unsupported @+1951319

I am not sure Sam still monitors this dash.org/forum, so pls inform him through other means about this issue and tell him that i started a conversation with him about this issue on dash.org/forum.
 
Last edited:
@pshenmic I see you found a possible fix to my problem : https://github.com/dashpay/platform/pull/1607

Dashmate load up and init WASM DPP on every command run, but in fact, it is only used in one place, where latest appVersion being extracted and set in local setup.

This causes cross-platform issues on some specific CPUs, more details could be read here #1583

This PR removes initialization of WASM DPP in the dashmate
Click to expand...

I just hope this does not mean i will have problems again with my specific CPU, once Platform gets released on Mainnet and Evonodes need to update to get access to both L1 & L2. Or can it be coded in such a way that WASM DPP initialization will not give cross-platform issues in the future ?

Also why no cross-platform issue with Dashmate v0.25.13 for my CPU ? What changed between v0.25.13 and v0.25.15 to affect cross-platform this way ? Are we sure that removing certain obsolete dependencies of Rust crates, did not cause the cross-platform issue in the first place ?
See : https://github.com/dashpay/platform/pull/1578
 
Last edited:
qwizzie said:
@pshenmic I see you found a possible fix to my problem : https://github.com/dashpay/platform/pull/1607



I just hope this does not mean i will have problems again with my specific CPU, once Platform gets released on Mainnet and Evonodes need to update to get access to both L1 & L2. Or can it be coded in such a way that WASM DPP initialization will not give cross-platform issues in the future ?

Also why no cross-platform issue with Dashmate v0.25.13 for my CPU ? What changed between v0.25.13 and v0.25.15 to affect cross-platform this way ? Are we sure that removing certain obsolete dependencies of Rust crates, did not cause the cross-platform issue in the first place ?
See : https://github.com/dashpay/platform/pull/1578
Click to expand...
No, removing obsolete dependencies not our case 100%, because this PR was merged into v1.0-dev, while we are working with 0.25 branch (master)

I don't think you should have these problems with dashmate, at least until we decide to get Wasm DPP back in, but there aren't use cases for that currently, and I doubt we will get them in the near future.

But what worries me even more, is that if you are able to reproduce it on your own, then it surely could also crash on someone else's system. Since you are using dashmate on the server and not a desktop system for an everyday use, this fix should be fine for you. But it looks like it can be possibly caught by random user on mobile or desktop, which is not good.

I made a sample application to reproduce your case, could you please clone this repo https://github.com/pshenmic/wasm-dpp-nodejs-load-sample anywhere on your system, and try to run it: `node index.js`

On my side, it loads successfully:
```
% node index.js
Wasm DPP successfully loaded
```
 
pshenmic said:
No, removing obsolete dependencies not our case 100%, because this PR was merged into v1.0-dev, while we are working with 0.25 branch (master)

I don't think you should have these problems with dashmate, at least until we decide to get Wasm DPP back in, but there aren't use cases for that currently, and I doubt we will get them in the near future.

But what worries me even more, is that if you are able to reproduce it on your own, then it surely could also crash on someone else's system. Since you are using dashmate on the server and not a desktop system for an everyday use, this fix should be fine for you. But it looks like it can be possibly caught by random user on mobile or desktop, which is not good.

I made a sample application to reproduce your case, could you please clone this repo https://github.com/pshenmic/wasm-dpp-nodejs-load-sample anywhere on your system, and try to run it: `node index.js`

On my side, it loads successfully:
```
% node index.js
Wasm DPP successfully loaded
```
Click to expand...
Cloned directly into my home directory.

~$ git clone https://github.com/pshenmic/wasm-dpp-nodejs-load-sample
$ cd wasm-dpp-nodejs-load-sample
$ npm install
$ node index.js

CompileError: WebAssembly.Module(): Compiling function #331:"memchr::arch::wasm3 2::simd128: packedpair::Find..." failed: Wasm SIMD unsupported @+1955693

Node.js v20.10.0
Click to expand...

Maybe the problem is with @dashevo/wasm-dpp (incorrect workspace ?) listed as dependencies for v.0.25.15 ?

Knipsel.JPG


Actual workspace : https://github.com/dashpay/platform/tree/v1.0-dev/packages/wasm-dpp ?

Even v0.25.16-rc.4 is currently linking to incorrect workspace @dashevo

Knipsel.JPG


But in any case, reproducing the same error on my side has been successfull with your sample application.
I will add this information to the platform github issue (https://github.com/dashpay/platform/issues/1583).

Update :

Knipsel.JPG
 
Last edited:
Hey folks, rc.6 is finally out and brings up few important fix and features. The most interesting feature a lot of you have been asking before, is to let dashmate update core image version itself by the command, and we eventually agreed to test this approach. Dashmate now contains tagged core docker image, the same as platform, allowing you to instantly upgrade your Core on the node as soon as new minor release comes out. Major core version upgrades (etc v19 -> v20) still requires a new dashmate release.


Dash Platform v0.25.16-rc.6

Noticeable changes:
* Allow to update minor Core versions via `dashmate update`
* Removes WasmDPP from dashmate (resolve possible cross-platform issues found by @qwizzie)
* Fix Dashmate Helper HTTP API (`command not found` issue)
* Added unit test for `dashmate update` command
* Added E2E test for Dashmate Helper HTTP API (ensures it is responsive in CI)
* Removed redundant `dashmate platform feature-flag` command

This release changes Core image update approach, and now you must issue `dashmate update` whenever new minor version comes out.


To try out this release candidate, simply do:
npm install -g [email protected]
dashmate update
dashmate restart
Click to expand...


.deb users, can just download a release from the github and omit npm install step
 
Last edited:
@pshenmic

I like this new approach to updating Dashmate. Looks better to me then for example using docker image tag 'latest', as i think
docker image tag '20' should be more informative about the version in use (v20), when issuing dashmate update command.

What is the difference between dashmate start and dashmate restart ? I normally use dashmate start instead of dashmate restart.
 
Last edited:
`dashmate restart` is just an alias for `dashmate stop && dashmate start`

I will check the `dashmate update`, possibly there is a chance to show what to what exactly version component have been updated
 
pshenmic said:
`dashmate restart` is just an alias for `dashmate stop && dashmate start`

I will check the `dashmate update`, possibly there is a chance to show what to what exactly version component have been updated
Click to expand...
this v0.25.16-rc.6 works for me.

Knipsel.JPG


I used a slightly different approach

dashmate stop
npm install -g [email protected]
dashmate update
dashmate start

Dashmate status will show exact Core version (v20.0.2), so this is fine with regards to v20
 
Last edited:
pshenmic said:
Hey folks, rc.6 is finally out and brings up few important fix and features. The most interesting feature a lot of you have been asking before, is to let dashmate update core image version itself by the command, and we eventually agreed to test this approach. Dashmate now contains tagged core docker image, the same as platform, allowing you to instantly upgrade your Core on the node as soon as new minor release comes out. Major core version upgrades (etc v19 -> v20) still requires a new dashmate release.


Dash Platform v0.25.16-rc.6

Noticeable changes:
* Allow to update minor Core versions via `dashmate update`
* Removes WasmDPP from dashmate (resolve possible cross-platform issues found by @qwizzie)
* Fix Dashmate Helper HTTP API (`command not found` issue)
* Added unit test for `dashmate update` command
* Added E2E test for Dashmate Helper HTTP API (ensures it is responsive in CI)
* Removed redundant `dashmate platform feature-flag` command

This release changes Core image update approach, and now you must issue `dashmate update` whenever new minor version comes out.


To try out this release candidate, simply do:



.deb users, can just download a release from the github and omit npm install step
Click to expand...
When I install the new .deb package, it says "downgrade." Is that just because of the way this one was named?
Code: 
dpkg: warning: downgrading dashmate from 0.25.16.56ce20547-1 to 0.25.16.1fa1ff2cc-1
 
@pshenmic

* New feature request for Dashmate *

Dashmate users have currently no way of knowing if their Masternode or Evonode is involved with 1 or more quorums / dkg sessions.
If Dashmate users do a dashmate stop or dashmate restart command due to a new Core update, they risk getting either a high PoSe score in case of being already involved with 1 quorum / dkg session or an instant PoSe ban in case of being already involved with 2 quorums / dkg sessions. I think the last is specially relevant to Evonodes.

I was wondering if Dashmate can check in the background for the Masternode or Evonode in question, if it is currently involved in 1 or more quorums / dkg sessions and interrupt a dashmate stop / dashmate restart command, if it detects such involvement ? Dashmate should then inform the user about the detected quorum involvement (+ associated risks of stopping Dashmate at that specific time) and refer to dashmate status command for more information and ask if user wants to continue with dashmate stop / dashmate restart yes or no.

Dashmate status command should be extended to show specific information about the detected quorums / dkg sessions (when they started and when they will end). The Dashmate user can then wait for an opportune moment (when the Masternode / Evonode is not involved in any quorum / dkg session), before issuing a dashmate stop or dashmate restart command and avoid a high PoSe score or an instant PoSe ban.

Update : looks like quorum information can also be found in the core.log / debug.log by searching on : 'quorum initialization OK for llmq_60_75' That quorum seems to last 31 consecutive blocks, meaning it counts from 0 to 31 and with a new quorum starts to count from 0 to 31 again.

If last entry was 'CDKGSessionManager::InitNewQuorum -- height[1992991] quorum initialization OK for llmq_60_75 qi[31]' and current block height is higher then height mentioned with quorum (1992991), then i think we can conclude our Masternode / Evonode is currently not involved in a quorum. At least not involved with a llmq_60_75 quorum.

There are two other types of quorums that can be found in our debug.log / core.log :

* quorum initialization OK for llmq_100_67 qi[0] : so this could be a second llmq quorum to take into account, but it always shows [0].
Maybe it lasts only 1 block?
* quorum initialization failed for llmq_50_60 qi[0] mns[0] : this llmq quorum always fails and also always shows [0]
 
Last edited:
Knipsel.JPG


Casually changing Dashmate update instructions without informing Dashmate users about this ?
Not cool.

So what happens when people issue npm update -g dashmate ? do they get a notice that they should be using npm install -g dashmate ?
With which Dashmate version does this change get active ? I did a npm update -g dashmate on my Evonodes yesterday without problem,
which got me to Dash Core v20.1 / Dashmate 0.25.22 so i assume this change gets active with Dashmate v0.25.23 ?

Or do commands npm update -g dashmate & npm install -g dashmate continue to coexist with each other ?
Which means Dashmate users can use either :

dashmate stop
npm update -g dashmate
dashmate update
dashmate start

or

dashmate stop
npm install -g dashmate
dashmate update
dashmate start

For a simple Dashmate update through npm, i would actually prefer to continue using npm update -g dashmate
For an uninstall and install of Dashmate through npm, npm install -g dashmate makes more sense (i am already using it that way, though i rarely needed to use it)
 
Last edited:
qwizzie said:
View attachment 12376

Casually changing Dashmate update instructions without informing Dashmate users about this ?
Not cool.

So what happens when people issue npm update -g dashmate ? do they get a notice that they should be using npm install -g dashmate ?
With which Dashmate version does this change get active ? I did a npm update -g dashmate on my Evonodes yesterday without problem,
which got me to Dash Core v20.1 / Dashmate 0.25.22 so i assume this change gets active with Dashmate v0.25.23 ?

Or do commands npm update -g dashmate & npm install -g dashmate continue to coexist with each other ?
Which means Dashmate users can use either :

dashmate stop
npm update -g dashmate
dashmate update
dashmate start

or

dashmate stop
npm install -g dashmate
dashmate update
dashmate start

For a simple Dashmate update through npm, i would actually prefer to continue using npm update -g dashmate
For an uninstall and install of Dashmate through npm, npm install -g dashmate makes more sense (i am already using it that way, though i rarely needed to use it)
Click to expand...

I don't remember if there are documents suggesting you to use `npm update`. The difference is that `npm install` is always picking the latest version of the package, while `npm upgrade` will perform some semantical versioning from your global package list. Those two commands do almost the same thing except npm install forcibly install you the latest version from the npm.
 
You must log in or register to reply here.
Back
Top