Dashmate (discussion)

[pshenmic]

Hey folks

I am opening this thread to collect all the questions about dashmate in one place, as we have tons of question all over the forum.

Feel free to ask anything, suggest idea or improvenment!

 

[qwizzie]

@pshenmic

Pls confirm from your own screenshot that 'dashmate status' command (the left part of your screenshot) is giving an output bug with both PoSe penalty (should show actual PoSe score, not n/a) and with Last paid time (1604.33 days is incorrect, should be Never).
It seems to me that you clearly reproduced those two output issues on your own test server. You can compare those two fields with the 'dashmate status masternode' command (right side of your screenshot) to see how those two fields should display their output correctly.

Other output issues i personally still have (for example Payment queue position and Next payment time), i assume will be solved for me when Dashmate v0.24.20 gets released and i update Dashmate to it.

With regards to 'Enabled count', i assume that is the number of enabled Masternodes (in your case the number of enabled Masternodes on Testnet). Should we maybe differentiate that count between enabled normal Masternodes and enabled Evonodes ? Just like how the Dash Core wallet differentiate between the two ? (although it uses HPMN instead of Evo)

I also wonder if Payment queue position should be based now on a combined count of enabled Masternodes and enabled Evonodes, as both of them now operate on L1 and share Payment queue position with each other. There are currently 52 enabled Evonodes operating on Mainnet, which means Payment queue position could be off by 52 blocks, if it does not take into account the Evonodes number ? I would like some confirmation if the Payment queue position logic needs correction or if it is working as intended.

 

[pshenmic]

Thank you very much, I've been really missing it, now I can see what you meant. I prepared a code change

With regards to 'Enabled count', i assume that is the number of enabled Masternodes (in your case the number of enabled Masternodes on Testnet). Should we maybe differentiate that count between enabled normal Masternodes and enabled Evonodes ? Just like how the Dash Core wallet differentiate between the two ? (although it uses HPMN instead of Evo)

That makes sense, created a task for it https://github.com/dashpay/platform/issues/1294
Enabled count right now should total enabled count (hpmn + regular)

I also wonder if Payment queue position should be based now on a combined count of enabled Masternodes and enabled Evonodes, as both of them now operate on L1 and share Payment queue position with each other. There are currently 52 enabled Evonodes operating on Mainnet, which means Payment queue position could be off by 52 blocks, if it does not take into account the Evonodes number ? I would like some confirmation if the Payment queue position logic needs correction or if it is working as intended.

Oh, I am not sure 100% because I did not write this payment calculation piece of code, but I feel like it should be okay. HPMN node is the same masternode, it just gets paid 4 times.

It does take a total enabled count as an argument, so I assume it should be correct.

PS:
We are still experience some infra issues in the CI but I hope I will be able to push new release as soon as possible

 

[qwizzie]

With regards to both the differentiation between number of enabled Masternodes and enabled number of Evonodes that you are planning for Dashmate (github issue https://github.com/dashpay/platform/issues/1294) and the payment queue position logic that should follow from that, please take note of the following :

HPMNs, Evonodes are paid 4 times in a row, so payment queue length should be number of enabled regular masternodes + (number of enabled HPMNs/Evonodes*4).

See : https://github.com/dashpay/dash/issues/5528

FYI.

 

[pshenmic]

Dashmate v0.24.20 was released

* Dash Core upgraded to v19.3.0
* Several dashmate status UI issues fixed (big ups @qwizzie for reporting)
* Dashmate Setup: stabilized external IP detection in some cases
* Dashmate Start / Stop / Reset: fixed --platform flag throw error (incorrect service filtering)

 

[qwizzie]

Dashmate v0.24.20 was released

* Dash Core upgraded to v19.3.0
* Several dashmate status UI issues fixed (big ups @qwizzie for reporting)
* Dashmate Setup: stabilized external IP detection in some cases
* Dashmate Start / Stop / Reset: fixed --platform flag throw error (incorrect service filtering)

@pshenmic

I am getting :

Error: (HTTP code 404) unexpected - manifest for dashpay/dashmate-helper:0.24.20 not found: manifest unknown: manifest unknown

after having updated dashmate through npm update -g dashmate
Dashmate throws that error after issuing command dashmate update and won't allow a dashmate start for me.

Is the dashmate helper updated through Docker as well ? If yes, then i don't see Dashmate v0.24.20 on Docker :

Latest update there seems to be dashmate v0.24.19 from 11 days ago.
Source : https://hub.docker.com/search?q=dashmate

My config.json (after issueing npm update -g dashmate) states the following at the very start :

{
"configFormatVersion": "0.24.20",
"defaultConfigName": "mainnet",
"defaultGroupName": null,
"configs": {
"base": {
"description": "base config for use as template",
"group": null,
"docker": {
"network": {
"subnet": "0.0.0.0/0",
"bindIp": "0.0.0.0"
},

I tried installing dashmate again through command npm install -g dashmate and then update it again with command npm update -g dashmate --> same error & same problem (will not allow dashmate start).

I tried installing dashmate again through command npm install -g dashmate, without issuing npm update -g dashmate (hoping that would get me to an older Dashmate version) --> same error & same problem (will not allow dashmate start).

Github issue : https://github.com/dashpay/platform/issues/1303

This needs urgent attention, as i can not start dashmate right now.

I would also like to know how i can revert back to a previous dashmate version (v0.24.19 in this case), in case of above situation occurring again in the future. So that i can at least start the dashd Docker container, while waiting for a solution.

 

[qwizzie]

Looks like a fix is in the works : https://github.com/dashpay/platform/pull/1304
I hope this gets merged fast, so that i can start my Evonode.

I also wonder if this just relates to Dashmate v0.24.20 or if it also applies to a fresh install of Dashmate v0.24.19
as the issue seems to be pointing to a compilation / build issue ?

Which could complicate any revert effort back to Dashmate v0.24.19 that i was thinking of.

Update : thanks to pshenmic i successfully reverted back to Dashmate v0.24.19 while fetching latest Core v19.3.0 update with commands :

npm install -g [email protected]
dashmate config set core.docker.image dashpay/dashd:19.3.0

 

[xkcd]

I hope this gets merged fast, so that i can start my Evonode.

Please note that you do not need Dashmate at all to start an evonode as can be seen from this article here.

How to create a no fuss Evo node in v19

Evo nodes aka High Performance nodes https://www.dash.org/hpmn-faq/ are Masternodes that will be required to run Platform services in v20 onwards, slated for release in December 2023*. This node type requires the collateralisation of 4000 Dash. During v19 the network will be in a transitory...

www.dash.org

 

[qwizzie]

My Evonode is currently already setup through Dashmate / Docker container, so at this point i am pretty much already committed to using Dashmate to maintain my Evonode (issue Dash Core updates etc) and i am also using this period to familiarize myself with Dashmate.

This should be helpfull to me when Evonodes get updated later on (most likely many more months 'later on', but we will see) to connect to both L1 & L2.

 

[xkcd]

i am also using this period to familiarize myself with Dashmate.

This is a very good reason to do so, however, it is my intention to support Evonodes on the DMZ if at all possible as the date (TBD) draws near.

 

[pshenmic]

New platform v0.24.21 was released that fixes CI infrastructure bug that failed to release dashmate v0.24.20 properly.

Please, update your dashmate nodes

PS:
I tested this one personally on the platform-explorer server. I was able to successfully update and restart my node:

npm install -g dashmate
dashmate update
dashmate restart

 

[qwizzie]

Successfully updated Dashmate to v0.24.21
Fixes look good.

Thank you.

 

[qwizzie]

@pshenmic

With Dash Core v20, sentinel functionality will be pulled directly into Core.
Below pull request is focused mainly on dashmate :

feat(dashmate): remove sentinel service by strophy · Pull Request #1354 · dashpay/platform

Issue being fixed or feature implemented Sentinel is now implemented as part of DashCore so sentinel service is deprecated and needs to be removed What was done? Remove sentinel Add deprecatedr...

github.com

It also removes the status of sentinel on dashmate status (and i assume also on dashmate status masternode).
So a simple npm update -g dashmate command would be enough to process these new changes for the docker system, once Dash Core v20 is released to Mainnet and dashmate has been updated.

I am not sure it also removes any obsolete sentinel installation data on those Masternodes / Evonodes, that are currently already setup through dashmate. Will that require a manual action for those users ?

I am also not sure how Masternodes/Evonodes which are not setup through dashmate, will handle sentinel functionality in Dash Core v20, while currently running a cronjob on restarting sentinel and having sentinel installation data present in their system thanks to :

git clone https://github.com/dashpay/sentinel.git
cd sentinel
virtualenv venv
venv/bin/pip install -r requirements.txt
venv/bin/python bin/sentinel.py

Will those users be required to manually delete their sentinel folder and delete their cronjob ?

So far the removal of sentinel on Masternodes / Evonodes through Github pull request seems mainly focused on dashmate setup, not dashmate setup & other forms of setup. Maybe the plan is to address other forms of setup and how to remove sentinel there, in the Dash Core v20 release notes?

 

[pshenmic]

Hey, qwizzie

To be honest, I'm not aware, I think thats the question for the core team, as I don't really even know what it was used for

I guess some kind of migration will be needed to move python sentinel into Dash Core or dashmate.

As there will be more info about that, I will be happy to add that support into the dashmate

 

[thephez]

I am not sure it also removes any obsolete sentinel installation data on those Masternodes / Evonodes, that are currently already setup through dashmate. Will that require a manual action for those users ?

I am also not sure how Masternodes/Evonodes which are not setup through dashmate, will handle sentinel functionality in Dash Core v20, while currently running a cronjob on restarting sentinel and having sentinel installation data present in their system thanks to :



Will those users be required to manually delete their sentinel folder and delete their cronjob ?

So far the removal of sentinel on Masternodes / Evonodes through Github pull request seems mainly focused on dashmate setup, not dashmate setup & other forms of setup. Maybe the plan is to address other forms of setup and how to remove sentinel there, in the Dash Core v20 release notes?

The Core RPCs used by sentinel were updated in the PR below so they will no longer interact with sentinel. Core will effectively ignore sentinel so governance won't be affected if someone doesn't explicitly disable sentinel. Disabling the cronjob would definitely make sense though.

feat(rpc): Disable submission of triggers and gobject vote-conf RPC by ogabrielides · Pull Request #5552 · dashpay/dash

Issue being fixed or feature implemented With #5525 , MNs shouldn't use Sentinel anymore. What was done? In order to force them to remove Sentinel: gobject submit RPC won't accept triggers anymore...

github.com

 

[qwizzie]

@pshenmic

I decided to do a bit of server updating and cleaning, so i did the following :

dashmate stop
sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade
sudo apt-get autoclean
sudo apt-get autoremove
sudo reboot now
dashmate start

After a few minutes and having verified in the core.log (debug.log) that sync is indeed finished, i tried some 'dashmate status' commands but i am not getting any specifics about Core and Sentinel anymore. I provided a screenshot for you to look at (blanked out some personal stuff) :

So this is all that i am getting returned, when doing a 'dashmate status' & 'dashmate status masternode' command after a successfull sync.
Dashmate still on version 0.24.21

I looked at https://www.npmjs.com/package/dashmate?activeTab=versions and noticed dashmate is currently on version 0.24.23
so i decided to stop dashmate once more and do a dashmate update through npm (npm update -g dashmate), but now i am getting a Docker Compose error when trying to stop dashmate :

dashd.pid stays active and mainnet node can not be stopped this way.

In the end i decided to do a npm dashmate update while dashmate is still active ('dashmate update' shows dashmate indeed updated to v0.24.23 this way) and then force a server reboot (so that dashd would be restarted again and dashmate would hopefully process the update changes, that it could perhaps not do while being active).

** Also be informed while doing npm update -g dashmate command, i got an alert from npm that npm itself is on v10.0.0 and users need to update to that, but as it turns out dashmate users can not update to npm v10.0.0 due to some conflict with v9 **

Unfortunetely even on dashmate v0.24.23 i can not stop dashmate manually, as it still triggers that same Docker Compose error.
And also dashmate even on v0.24.23 is not showing Core and Sentinel details with commands 'dashmate status' & 'dashmate status masternode'.

So to summarize :

I have dashmate running, but i can not stop it through a 'dashmate stop' command (i get a Docker Compose error)
Some alert about npm on v10.0.0 but unable to update npm to that version.
Dashmate status commands do not show Core & Sentinel details anymore, even on latest dashmate version.
(it is almost like dashmate thinks it is still in sync phase)

 

[pshenmic]

Ohhh, thats unfortunate. Might be something happened during upgrade? v0.24.21-v.0.24.23 doesnt really introducing any major changes, there is a little things for a tenderdash to be able to run in seed mode for devnet / testnet. What is your current nodejs / npm version?

 

[qwizzie]

Ohhh, thats unfortunate. Might be something happened during upgrade? v0.24.21-v.0.24.23 doesnt really introducing any major changes, there is a little things for a tenderdash to be able to run in seed mode for devnet / testnet. What is your current nodejs / npm version?

It would be interesting to see if there is a difference between my npm version (and dependencies) and your npm version (and dependencies).
It would also be interesting to see if you encounter the same problem, after doing the server maintenance and cleaning steps that i mentioned in my previous post.

Any idea why dashmate throws that Docker Compose error : cannot read properties of undefined (reading 'trim') ?
That error looks a bit like that other Docker Compose error we had in the past and that got fixed in a dashmate update.
I am on Ubuntu 20.04.6 LTS (GNU/Linux 5.4.0-162-generic x86_64).

 

[xkcd]

This is horrifying, all those dependencies just to install a freaking dashd? Have you heard of the Dash Masternode Zeus?

 

[pshenmic]

Any idea why dashmate throws that Docker Compose error : cannot read properties of undefined (reading 'trim') ?
That error looks a bit like that other Docker Compose error we had in the past and that got fixed in a dashmate update.
I am on Ubuntu 20.04.6 LTS (GNU/Linux 5.4.0-162-generic x86_64).

Hey there, I just have reproduced the same issue on my platform explorer production server. After figuring it out, it turned out docker-compose-plugin 2.21.0 introduces breaking change , that is not compatible with dashmate

Bug fixes and enhancements link​

• Changed docker compose ps and docker compose ps --format=json output to align with Docker CLI.

As a temporary fix, please use this command to install previous docker-compose-plugin release:

sudo apt-get install docker-compose-plugin=2.20.2-1~ubuntu.22.04~jammy

I will prepare all necessary changes in the next version to properly handle that.

This is horrifying, all those dependencies just to install a freaking dashd? Have you heard of the Dash Masternode Zeus?

I don't catch you at all. These dependencies are internal stuff of the Node.JS, that does not even required to run dashmate (since you can install it simply with .deb)
The only required dashmate dependency is Docker (which I believe also handled if you install with .deb)

Please stop blaming the product without reason, thats upsetting