Dash BIP32 serialization values [Dev discussion - won't apply to most]

nmarley

nmarley

Administrator
Dash Core Group
[TL;DR]: We should change "drkv/drkp" to something more Dashy, oh, and there's already problems in the Dash software ecosystem that will get worse if a solution isn't agreed upon.

BIP32 keychains can be serialized for human-readable use. They come in 2 flavours, "public" and "private".

For Bitcoin, the public ones look like: "xpub....blajbalbalblabla" and the private ones look like: "xprv...blablbalbah". Dash has slightly different values.

People build Bitcoin/Dash/etc software based on these assumptions which are standardized in the BIP0032 text and in the altcoin reference client code. (E.g. these are defined in dash/src/chainparams.cpp).

Dash's public ("xpub") value is currently: "drkv" (according to the reference client). The private ("xprv") value is "drkp" (again, this is according to the code I mentioned above).

moocowmoo Came up with the values, but with one difference - the values he came up with were so that "drkp" means public and "drkv" means private. They got swapped when they were added to the reference client. He's added his (correct) version of those values to pycoin (A Bitcoin library for Python) and is using them in the dashvend code also.

So we've got some software implementations (including the pycoin library) which implement these values swapped from the reference client.

[Note: I'm not saying moocowmoo is doing it "wrong". He invented these values, so if anything, the ref client is "wrong". It's not about "wrong", it's about how to get these all in-sync so that future software developed for Dash doesn't run into confusion and problems.]

Another problem is that the Electrum-Dash client is using "xpub" and "xprv" (Bitcoin's version bytes) and that's going to cause problems with software interoperability down the road.

So there's already multiple conflicting implementations.


In addition to all that, the current values are one of the last vestiges of the old "Darkcoin" days. I think now would be a good time to consider changing the drk* prefixes to something more Dash-y.

Me, I've been porting different software versions using the ref. client values b/c that's what the "reference" client is supposed to be — a reference.

But as more software gets developed using different starter templates (e.g. the vending machine python code), this will grow to become more of an issue.

If we can make a clean break to more "Dashy" version values, announce and change all projects now, it might be easier than trying to fix the resulting mess down the road. New software keeps being created (good), and that's what we need, but I can foresee a mess if we don't get things cleaned up at this point. As we all know, BIP32 isn't a small thing, it's huge for payments acceptance and ease-of-use for "normal" users who just want a simple way to back up and secure their coins.

Normally this wouldn't be a problem, but a lot of the software actually uses the header byte versions for things like determining mainnet/testnet and whether it's a valid xpub key or not. So it *is* kinda important to be on the same page with these.


Developers, any thoughts? Can we maybe converge on a solution and implement that in all software to prevent problems in the future?
 
Sidenote: I was about to copy my reply from our recent discussion on Slack but it's already gone... /sigh/ I'll try to recreate my flow of thinking here as much as I can to stay consistent :)

I like the idea, drkp/drkv was always confusing for me anyway - can't quickly say which one is public and which one is private because of that p at the end (sorry, moocowmoo :rolleyes:).

If we are going to change this I would consider smth more explicit and easily readable like it's in bitcoin. My guesses are:
- xdpu/xdpr or xdpb/xdpr
which is still close to the way it's in bitcoin but way more explicit imo
- dpub/dprv
which is close to bitcoin but from the other side :grin: yes, I know x stands for "extended" there but we can think of it like "dash extended"

Anyway, I think we really should clean that mess you just described.

edit: pinging eduffield flare crowning
 
UdjinM6 said:
Sidenote: I was about to copy my reply from our recent discussion on Slack but it's already gone... /sigh/ I'll try to recreate my flow of thinking here as much as I can to stay consistent :)

I like the idea, drkp/drkv was always confusing for me anyway - can't quickly say which one is public and which one is private because of that p at the end (sorry, moocowmoo :rolleyes:).

If we are going to change this I would consider smth more explicit and easily readable like it's in bitcoin. My guesses are:
- xdpu/xdpr or xdpb/xdpr
which is still close to the way it's in bitcoin but way more explicit imo
- dpub/dprv
which is close to bitcoin but from the other side :grin: yes, I know x stands for "extended" there but we can think of it like "dash extended"

Anyway, I think we really should clean that mess you just described.

edit: pinging eduffield flare crowning
Click to expand...
From QM perspective i like dpub/dprv most, as it is very explicit about what is what. Serializations should be best readable by humans, machines do not misread strings :)

So if we want to finally fix this, I'd say lets go for something from "apub" to "Zpub"
 
UdjinM6 : I use the mnemonic "There's no v in public" to easily remember what's what.

Granted, all these choices are compromises. Four characters is limiting.

I was working with pycoin and saw coins with alternate prefixes, so I calculated some for some of my favorite coins I had at the time and submitted a pull request along with this proposed standard:

Code: 
- suggestion for alt-coin bip32 prefix standardization

use first three characters of commonly accepted trading symbol with
suffixes v and p for private and public respectively.
two character symbols use suffixes pv and pb.
one character symbols use suffixes pub and prv.

testnet addresses are to use all capital letters

See the discussion from 2014 here: https://github.com/richardkiss/pycoin/pull/51

Regardless of what we choose, If the network intent is known, converting header bytes is trivial (xpub -> dshp -> xpub)
nmarley wrote this ruby to convert headers https://gist.github.com/nmarley/506a3d3036d55c878696
 
Last edited by a moderator:
moocowmoo said:
UdjinM6 : I use the mnemonic "There's no v in public" to easily remember what's what.

Granted, all these choices are compromises. Four characters is limiting.

I was working with pycoin and saw coins with alternate prefixes, so I calculated some for some of my favorite coins I had at the time and submitted a pull request along with this proposed standard:

Code: 
- suggestion for alt-coin bip32 prefix standardization

use first three characters of commonly accepted trading symbol with
suffixes v and p for private and public respectively.
two character symbols use suffixes pv and pb.
one character symbols use suffixes pub and prv.

testnet addresses are to use all capital letters

See the discussion from 2014 here: https://github.com/richardkiss/pycoin/pull/51

Regardless of what we choose, If the network intent is known, converting header bytes is trivial (xpub -> dshp -> xpub)
Somebody already wrote some python to convert (and recalc checksums) as needed, but I can't remember who nmarley ? or where that gist is right now.
Click to expand...
To follow your instructions exactly it would be dasp/dasv ;)
I don't know... I don't "feel" it
 
cross posting from slack:

I don't really think prefix selection is a huge deal as long as there's a way to force a network context for a seed to be interpreted under. (for instance, pycoin has an --override-network switch)

My prefix standardization proposal isn't ideal, but there's only 4 characters to make use of, and nobody else has come up with any other suggestions that might apply to all coin types.

Even though the seed itself is network agnostic, I think it's a good idea to know which network the seed was generated for/is used by, which is why I promoted the idea in the first place.

If network metadata belongs in the serialized bip32 string, maybe the standard should be extended to support better labeling.

Also, dashvend is using pycoin, so I leveraged its interpretation of header bytes as network identity, which I don't think is an established standard anywhere, rather a convenience.

So, since all seeds (regardless of prefix) will generate the same addresses within a selected network the only thing that might break interoperability is a developer requiring or testing for a particular bip32 prefix (like xpub and xprv) -- which they probably shouldn't considering the string is checksummed.
 
moocowmoo said:
So, since all seeds (regardless of prefix) will generate the same addresses within a selected network the only thing that might break interoperability is a developer requiring or testing for a particular bip32 prefix (like xpub and xprv) -- which they probably shouldn't considering the string is checksummed.
Click to expand...

It's not a big deal, except that libraries and software are being written (and already written) which are based on that assumption -- here are some examples:

  • Oleg's BTCRuby library, which Mycelium Gear is based on.
Those are the ones that I have recent experience with, but I'm sure many others do the same.

Personally, I'm aware of the issue -- but as a developer, I'll have to write a translation layer now to make up for it, instead of just relying on the library.

Also, there's no way I can convert key in one serialization format to another and display that to the user — it would confuse the heck out of them to have their "xpub" transformed into "drkv". So I'll have to display the xkey that they entered, but silently convert it in the background, for use with BIP32 libraries. Obviously that's confusing and not ideal, especially for newer software developers who are not aware of the issue.


Regarding the values, I think "dpub/dprv" is extremely clear which is which. :grin:
 
I think dpub/dprv will work fine. Their magic numbers are 0x02fda923 and 0x02fda4e8

I've never liked dropping the pub/prv suffixes, but was compromising to make the available namespace as large as possible.

What do you suggest for testnet values?
 
Last edited by a moderator:
moocowmoo said:
I think dpub/dprv will work fine. Their magic numbers are 0x02fda923 and 0x02fda4e8
Click to expand...

Out of curiosity, how do you know which values which will serialize to the correct strings? I tried guesswork/experimenting, but didn't get it quite right.
 
I manually search for good values. The trick is tight iterations. I set up vim to invoke setup install and ku on buffer write so I can check a few dozen values a minute. Then I just binary search each position to narrow down the prefix value and follow up tweaking the least significant values until the characters after the prefix balance visually.

Here's what I came up with (and some alternate values for mainnet):

Code: 
# mainnet - digit boundary after prefix
dpub8Du2vbh7JUQr  0x02fda923
dprv3hCznBesA6jB  0x02fda4e8

# mainnet - mostly uppers after prefix
dpubZ9169KDAEUny  0x02fda926
dprvLJcNFzfZn6yw  0x02fda4ea

# testnet - all lowers after prefix
DPUBxioaFQK3tpXx  0x3a59ea14
DPRVxvabDoWZUz2z  0x3a58f342
 
dpub/dprv (Any magic numbers resulting in those strings) and DPUB/DPRV look fine to me for mainnet and testnet. Any intentions of opening a PR to the reference client?
 
Thanks for pointing this information out, in the Dash Wallet for android, I am still using the Bitcoin values for these two parameters.
 
HashEngineering said:
Thanks for pointing this information out, in the Dash Wallet for android, I am still using the Bitcoin values for these two parameters.
Click to expand...

Well Electrum's still using the Bitcoin values too. I almost want to just shrug at this point. For interoperability, I've just been ignoring the version bytes and checking the priv/pub bytes instead, then I'll re-serialize it for the library I'm using.

E.g. if the user enters an "xpub" key, but the BIP32 library requires "drkv", I just shim it in.
 
You must log in or register to reply here.
Back
Top