Pre-proposal: Pythonic language Dash implementation

[priestc]

I saw Evan Duffield's presentation at the Dash Open House, and in that presentation he said there needs to be a "more simplistic, easier to understand implementation of dash". This describes exactly my project moneywagon, which you can see here:

https://github.com/priestc/moneywagon

I am taking a "lightweight first" approach. You can use moneywagon today to build a lightweight wallet (which is exactly what MultiExplorer is). The next step is making moneywagon capable of being a full node. Then after that, make it so you can mine dash with moneywagon.

I want to make a dash treasury proposal to get funding to get moneywagon to a point where it can be a fully validating node. I need the budget funds so I can work on moneywagon full time. For the past 2 years I've been working on moneywagon as a weekend-only side project.

What details do I need to include in my proposal?

 

[AnarchicCluster]

Interesting proposal. I guess members of the core team should express their opinions here since they are the ones most knowledgable about coding. They have the expertise to evaluate this project.

What kind of money are you looking for?

 

[demo]

Interesting proposal. I guess members of the core team should express their opinions here since they are the ones most knowledgable about coding. They have the expertise to evaluate this project.

The ones who are knowledgable about coding cryptography was/is the Satoshi. All the rest just copied the initial satoshi bitcoin code, and they are afraid to touch it. This is why the development of the Bitcoin core code is so slow. They havent even manage to upgrade the berkeley db of the bitcoin code, it remains the version 4.8.30 that dates back to 2010 (the latest version of berkeley-db is 6.x.x).

The Dash core developers also, they reside only in the surface of the bitcoin code, and they dont touch the bitcoin's core. This is the reason they invented sentinel and they store the votes in an external rdbms database, it is because they are unable to hack bitcoin's core code and create a decent vote blockchain (there is also another reason they chose sentinel solution, they wanted the dash specific govenrance code to be isolated from bitcoin core because they are afraid that the core developers of bitcon may sabotage the Dash specific code, by adding on purpose some dash incompatible code chunks into the main bitcoin core code)

So I dont think the current Dash core developers are capable to implement a bug free code of Dash, from scratch. With a single exception maybe, @UdjinM6. But he will need a lot of time (and probably a lot of money) to do this, with the final results being doubtful.

This is the sad truth. But I hope and I wish that someone will be able to built Dash and a decent votechain from scratch, and do it in a bug free way. This is needed desperately, dash needs to be detached from bitcoin.

 

[demo]

I saw Evan Duffield's presentation at the Dash Open House, and in that presentation he said there needs to be a "more simplistic, easier to understand implementation of dash". This describes exactly my project moneywagon, which you can see here:

https://github.com/priestc/moneywagon

I am taking a "lightweight first" approach. You can use moneywagon today to build a lightweight wallet (which is exactly what MultiExplorer is). The next step is making moneywagon capable of being a full node. Then after that, make it so you can mine dash with moneywagon.

I want to make a dash treasury proposal to get funding to get moneywagon to a point where it can be a fully validating node. I need the budget funds so I can work on moneywagon full time. For the past 2 years I've been working on moneywagon as a weekend-only side project.

What details do I need to include in my proposal?

I dont think that python language is a suitable language for the needs of Dash.

You should use another language. For example Tezos team chosed Ocaml, for the below reasons.

OCaml's static type system can help eliminate problems at runtime. However, it also forces the programmer to conform to the constraints of the type system, which can require careful thought and close attention. A type-inferring compiler greatly reduces the need for manual type annotations. For example, the data type of variables and the signature of functions usually need not be declared explicitly, as they do in Java. Nonetheless, effective use of OCaml's type system can require some sophistication on the part of a programmer.

OCaml is perhaps most distinguished from other languages with origins in academia, by its emphasis on performance. Firstly, its static type system prevents runtime type mismatches, and thus obviates runtime type and safety checks that burden the performance of dynamically typed languages, while still guaranteeing runtime safety, except when array bounds checking is turned off, or when some type-unsafe features like serialization are used. These are rare enough that avoiding them is quite possible in practice.

My POV is that you should also use a language (or a method) that suppots Reflection. Even better use a natively reflective language. The reflection behavior is what @eduffield is trying to imitate, with the use of Sporks.

@flare and @AnarchicCluster thanks for the troll rating, I was stuck in 666 troll rates and I didnt like that

 

[demo]

So, in order to support the reflection property, maybe you should re-write all of you code as follows:

#Python without reflection
obj = Foo()
obj.hello()

# Python with reflection
class_name = "Foo"
method = "hello"
obj = globals()[class_name]()
getattr(obj, method)()

# with eval
eval("Foo().hello()")

More info on python reflection can be found here:
https://en.wikibooks.org/wiki/Python_Programming/Reflection
https://docs.python.org/2/c-api/reflection.html


You can also do static type checking in python, there is an experimental static type checker that can be found here http://mypy-lang.org/


Furthermore your python code should pass a static program analysis, for performance issues and bug discoveries.
http://stackoverflow.com/questions/...yze-python-code-to-identify-problematic-areas


All of the above is the easy part. The difficult part is to discover potential logical bugs in your code, and this cannot be solved whatever automation tool or checker you are using, and whatever language.

This needs a lot of eye balls to watch your code. "given enough eyeballs, all bugs are shallow" How many eyeballs have already watched your code?

 

[UdjinM6]

Interesting approach. So instead of using your own node/backend you're using a bunch of external services. This indeed helps to get started but I guess won't scale if/when your tool/wallet would get some noticiable user adoption so turning it into fully validating node which connects to p2p directly probably would make sense if it would be a tool/wallet for a single coin. I'm not sure how you are going to implement all these different fullnodes in one though, this might be not so much realistic goal to accomplish imo. So I would suggest next step to be not a full node but a lightweight desktop wallet like Electrum, some users probably would like to have smth like that and this could be especially helpful for coins with no Electrum servers.


Btw, it seems like you to have quite a good knowledge in underlying tech and python so... maybe you can join and help us building Evo (tools?) instead?
I'm not a python/evo guy though, so pinging @nmarley @chaeplin @akhavr @snogcel @AndyDark here to chime in

 

[akhavr]

I've glanced over it my morning, didn't get it, so subscribed to this thread. Perhaps wasn't caffeinated enough, will revisit tomorrow.

Original post as well as 1st paragraph at https://github.com/priestc/moneywagon doesn't make bit sense to me. Why SPV is outdated, and what's the goal of "Blockchain Kernel"? Just framework for lightweight wallets or something bigger?

Tnx

 

[camosoul]

Interesting approach. So instead of using your own node/backend you're using a bunch of external services. This indeed helps to get started but I guess won't scale if/when your tool/wallet would get some noticiable user adoption so turning it into fully validating node which connects to p2p directly probably would make sense if it would be a tool/wallet for a single coin. I'm not sure how you are going to implement all these different fullnodes in one though, this might be not so much realistic goal to accomplish imo. So I would suggest next step to be not a full node but a lightweight desktop wallet like Electrum, some users probably would like to have smth like that and this could be especially helpful for coins with no Electrum servers.


Btw, it seems like you to have quite a good knowledge in underlying tech and python so... maybe you can join and help us building Evo (tools?) instead?
I'm not a python/evo guy though, so pinging @nmarley @chaeplin @akhavr @snogcel @AndyDark here to chime in

This is essentially what I was doing a year ago, but for vendors, before the frivolous lawsuit threats were started... Just chaining together existing services.

The work is basically done already. DASH just doesn't want it.

 

[priestc]

@akhavr "Blockchain kernel" was a bit of terminology I made up to describe what moneywagon does. Peter Rizun invited me to give a talk at one of his conferences where I explained hat Blockchain Kernel is all about. For some reason they never published the video, but you can read the slides here: https://docs.google.com/presentation/d/1EfaPmtB0qy8ES4z6vx9ylgjVSaJTHYX5Z7cNCdiW-V4/edit?usp=sharing

What kind of money are you looking for?

I just want what all the other developers get.

I'm not sure how you are going to implement all these different fullnodes in one though, this might be not so much realistic goal to accomplish imo.

I disagree. Litecoin is 99% bitcoin, as is Dash, as is Dogecoin, etc. If you build a dash full node, you've already built 99% of a litecoin node. It only makes sense to build a full node implementation that supports multiple currencies (as long as one stays away from non-bitcoin forked currencies such as NXT, Monero and ETH).

Take a look at this: https://github.com/priestc/moneywagon/blob/master/moneywagon/crypto_data.py#L319

Each currencies is defined by it's differences from bitcoin. In the future one could use moneywagon to launch a new cryptocurrency without ever needing a C++ implementation. All they would have to do is add an entry to "crypto_data" that defines it's parameters.

So I would suggest next step to be not a full node but a lightweight desktop wallet like Electrum, some users probably would like to have smth like that and this could be especially helpful for coins with no Electrum servers.

That is essentially what https://multiexplorer.com/wallet is. (the block explorer part of multiexplorer is terrible, but the wallet is much more polished)

I dont think that python language is a suitable language for the needs of Dash.

I disagree. Any turing complete language is capable of doing what any other turing complete language can do. All cryptocurrency is is calculating numbers. Stuff like typechecking is just a matter of personal preference.

 

[UdjinM6]

...
I disagree. Litecoin is 99% bitcoin, as is Dash, as is Dogecoin, etc. If you build a dash full node, you've already built 99% of a litecoin node.
...
Take a look at this: https://github.com/priestc/moneywagon/blob/master/moneywagon/crypto_data.py#L319

Each currencies is defined by it's differences from bitcoin. In the future one could use moneywagon to launch a new cryptocurrency without ever needing a C++ implementation. All they would have to do is add an entry to "crypto_data" that defines it's parameters.
...

Well, I agree about Litecoin and Doge but I guess you need to dig deeper into Dash code cause this part is a bit stretched imo - you are only implementing stuff we inherited from bitcoin and ignoring all Dash-specific protocol additions so in general this statement is simply false.

...
That is essentially what https://multiexplorer.com/wallet is. (the block explorer part of multiexplorer is terrible, but the wallet is much more polished)
...

Only that it's not. A wallet is a "key chain" for multiple addresses but the one you referenced is simply a representation of a single address (which hold no private keys).

 

[priestc]

ignoring all Dash-specific protocol additions

Not so much "ignoring" but rather "not gotten to yet". I have been focusing my efforts on getting the base BTC functionality working, as that work applies to other coins forked from BTC as well. If dash pays me, I'll be glad to add "instant send" and "private send" to multiexplorer. Adding dash specific features to multiexplorer is another treasury idea I have, in addition to the one in the op.

Only that it's not. A wallet is a "key chain" for multiple addresses but the one you referenced is simply a representation of a single address (which hold no private keys).

When I say wallet I mean "software that send and receives cryptocurrency", which is exactly what multiexplorer wallet does. It also does not use single addresses, it uses a new address for each change address, and it conforms to the BIP44 spec.

Fun fact, you can copy+paste a mnemonic from Dash Evolution into multiexplorer wallet, and your dash balance should show up in your multiexplorer wallet. (Vise versa as well). This is assuming Dash Evolution conforms to BIP44... The same goes for Copay. If you copy+paste your copay mnemonic into multiexplorer, your BTC balance will show up. You can also use the same mnemonic to store BTC and Dash, and they both will show up in multiexplorer.

 

[JZA]

Yes more development

There is a full pythonic node for Bitcoin here:
https://github.com/samrushing/caesure
https://github.com/ricmoo/pycoind
Also a recent NodeJS implementation that has been builted by a PurseIO developer:
https://github.com/bcoin-org/bcoin

Sent from my MotoE2 using Tapatalk

 

[UdjinM6]

...
When I say wallet I mean "software that send and receives cryptocurrency", which is exactly what multiexplorer wallet does. It also does not use single addresses, it uses a new address for each change address, and it conforms to the BIP44 spec.
...

Hmm.. weird... At the time I open it last time it only gave me an option to "enter any cryptocurrency address" and no register/login... Now I see this additions.. Well, maybe I was blind

PS. now I can't register...

 

[priestc]

PS. now I can't register...

Are you using Safari? There is a bug in bitcore-mnemonic that only effects Safari. I just tried registering a new account on Chrome and it worked.

 

[demo]

I disagree. Any turing complete language is capable of doing what any other turing complete language can do. All cryptocurrency is is calculating numbers. Stuff like typechecking is just a matter of personal preference.

Yes of course, but you forgot the security issues. A high level language, especially a language that often requires automatic packages updates from the network, is always susceptible to security bugs. And a program written in that language can even be compromised for specific targeted IPs either by the inventor of the language or by those who control the transport layer during the update. A low level language is always more secure, as long as it is a stable executable and its behavior do not depends on network updates.

What language do you consider to be more secure, an interpreted or a compiled one? It is obviously the compiled one, as long as the executable is always low level. In an interpreted language someone may change in the runtime the way the language interprets something, and this danger occurs whenever a network update is available in the packages of the language.

Python is a bad choice for this very reason. You may code a totally secure python program, but the language creators may target your IP and send you a compromised package through a network update, and that way change your program's behavior. Or maybe they didnt wanted to do that, but some agents forced them. We are talking about money, so security is a very imporant issue. We trust nobody, not even the language and package creators.

Despite all the above, Dash developers chose python language for their sentinel. Their choice is a generic strategical choice, they did this for the same reason they selected static IPs for the masternodes. Some people want Dash network to be able to shutdown in three clicks, in case of emergency.

 

[UdjinM6]

Worked in Chrome, thanks.
Now I can't send funds out

NoService: Could not push this dash transaction.! Tried: Siampm Dash Insight -> Skipped: parameter 1 must be hexadecimal string (not 'send')

 

[demo]

https://multiexplorer.com/wallet/

How can I register? I enter a username and a password (i ignore the optional fields) and I push the register button.
Then I go to login, but it fails.

I have firefox.

Ok...I see. Only Chrome works.

 

[priestc]

Worked in Chrome, thanks.
Now I can't send funds out

NoService: Could not push this dash transaction.! Tried: Siampm Dash Insight -> Skipped: parameter 1 must be hexadecimal string (not 'send')

That error is coming from a bitpay insight instance that was modified to work on dash:

https://github.com/UdjinM6/insight-api-dash

It appears you are the person who made these modifications. I assume you are the person who also runs http://insight.dash.siampm.com? Do you know why the error is happening? It used to work for me. Something has changed with the API in the past 6 months or so? Did you disable pushtx by any chance? In the docs it says to POST the rawtx to the url "/api/tx/send", which is exactly what multiexplorer is doing. It seems that your insight instance is trying to interpret "send" as a txid and it thinks I want to get TX info.

https://multiexplorer.com/wallet/

How can I register? I enter a username and a password (i ignore the optional fields) and I push the register button.
Then I go to login, but it fails.

I have firefox.

Ok...I see. Only Chrome works.

It should work for Firefox too. Do you get an error message?

 

[UdjinM6]

That error is coming from a bitpay insight instance that was modified to work on dash:

https://github.com/UdjinM6/insight-api-dash

It appears you are the person who made these modifications. I assume you are the person who also runs http://insight.dash.siampm.com? Do you know why the error is happening? It used to work for me. Something has changed with the API in the past 6 months or so? Did you disable pushtx by any chance? In the docs it says to POST the rawtx to the url "/api/tx/send", which is exactly what multiexplorer is doing. It seems that your insight instance is trying to interpret "send" as a txid and it thinks I want to get TX info.
...

Ah, I see. No, I'm not the person running it, it's @thelazier, and no, I didn't modify that part. Given the error message it looks like you are using GET instead of POST.

 

[priestc]

I'm definitely using POST. Here is the code: https://github.com/priestc/moneywagon/blob/master/moneywagon/services.py#L1207

This was working perfectly 6 months ago, so something had to have changed between then and now. By the way, this is only problm because there is only one service defined for Dash in moneywagon. This problem would never happen with bitcoin or litecoin because both of those currencies have multiple services listed for "push_tx". Moneywagon uses a fallback mechanism to try another service if one service is down or is broken somehow.

BTW, your coins are not stuck, if you open the dev console in your browser, you can see the raw tx printed to the console which you can then copy+paste into your dash full node, and from there it will make its way onto the network.

Also, I think this whole situation illustrates an attribute of moneywagon/multiexplorer that I designed into it and that is "always show the most verbose error message to the user". I absolutely hate it when software tells you "oops an error occurred", or "whoopsie something went wrong". I think it's much better to tell the user as much as the program knows in terms of what actually went wrong. That way I can diagnose the problem quickly and issue a fix very fast. If all that ever came up was "oops something went wrong" or some other folksy message, then thats not much to go off, and the problem may never get fixed.

You may code a totally secure python program, but the language creators may target your IP and send you a compromised package through a network update, and that way change your program's behavior.

All of the problems in your post can be worked around. First off, moneywagon/multiexplorer does not require "automatic packages updates from the network". There is nothing stopping someone from using Multiexplorer on their own machine running the same version for years to come. The only time I update the code is to add new feature, I hardly ever have to change existing code. If you want to be highly paranoid about security, you can just not ever update python or Multiexplorer or any of it's dependencies. The one exception is the "services" part of moneywagon, which does have to be updated frequently because services change their url, tweak their API, and go offline and stuff like that.Currently all services are defined in the "services.py" file that is linked at the top of this post. That file could be moved out to it's own package and updated on a different cycle as the rest of moneywagon. The services.py file is very "shallow" in the sense that any college freshman who knows python can code review that file and determine nothing malicious exists. You are right someone can sneak malicious code into python packages, but there are ways to mitigate the risks.