J2 RF 04 Jason Carver state of python ethereum tooling

des alleged in a pop-up on say support files said doozy m-files so sorry I won't try anymore French I promise so we're here for the state of Python tooling my name is Jason Carver I'm a web 3 PI contributor and we're gonna be talking about a whole lot of the state of Python on ethereum this was my first title I'm not good at titles it's a brief in completes game across the surface of Python tooling on etherium because there's way too much stuff out there to cover so my apologies in advance for the many things that I'm leaving out so before I get to exactly where we are right now I figure we should start with where we ought to be where we'd like to be and so I think this xkcd comic really captures where what I love about the best Python libraries that are out there that you can just import anti-gravity and fly so that's where I think we ought to be before I continue on if anyone is like tempted to take pictures during or worried about getting slides this is a QR code to the slides as well as the URL for it so I'll pause for just a second for anyone who wants to and grab that URL a QR code so that people can be engaged it looks like we are okay so what is what does import anti-gravity mean for etherion so we'd like a whole ecosystem of tools that are pythons all the way down so we want to be able to import feature libraries that do exactly what we want those future libraries should probably be depending on you know underground libraries that do one thing well and we wanted to be a Python all the way down so that we can hack and understand the whole the whole stack so a brief list of some examples of things that we would like to be able to do with Python so you know if your transaction is stuck on because your gas price wasn't high enough it should be a one-liner to bump up the gas price of a pending transaction you know it should be quick and easy to open state channels or make payments or verify payments micro payments across them you know we might want to do Hardware it use hardware wallets to do all of the things and we want to easily verify contract sources locally so I'm going a little bit more into each of those further on so how close are we unfortunately not too close we have we have a little ways to go but we're on the right path so there's a lot of things to work on the of course there's you know we can't work on everything at once so we'll just pick out a few things to talk about I think will be most impactful first so we've got the node interface non hosted keys verifying contract source and all Python node these are things that we think will have the most impact early on and well I'll go into each of them in detail here so we'll start with the node interface so what I mean by the the node interface for people who are perhaps just getting started a quick like reminder of a you know simplified version of how the stack works so if you've written a DAP where people can visit in and missed that DAP then talks to the theorem blockchain over web 3GS so it imports web 3 Jas uses which then uses the JSON RPC API to talk to your node maybe get their parity or something else and then uses and then that node of course connects to the rest of the network for for you know broadcasting your transaction or retrieving the data or whatever so what does that mean for Python instead so essentially the same practice here so we've got instead of web through Jas we've got web through PI and you know your Python app is written on top of that so when we talk about the node interface we're talking about you know essentially everything that you want to do with the network itself is going through web 3 and and so that is your interface to to everything so it seems like a good place to start if there are any problems there and there are a few things that are pretty rough around the edges so what's weighing us down lots of things choose a couple you know currently web 3 pies in version 3 stable right now so this is when I say what's current that's what I'm talking about the stable version you know simple things typos and destination addresses on is gonna cause an unrecoverable loss of funds you know you mix up an a4 a4 and it's just gone you know deciding which gas price to use it's you know tricky enough ish when you're doing it manually but if you have a long-running script you better be pretty good at automatically picking a gas price if you're gonna end up with a lot either overpaid transactions or transactions they get stuck pending integrating too you know using contract calls without hosted keys that'll make a little more sense in the next session or the next section and the data types were not very standard so you know if you were getting if you were interacting with the contract that returned bytes you're getting a string back that was hex encoded and sometimes it was expecting when you were sending bytes to send and code and bytes it was it was unpredictable that's that's so that is the current state so some examples of things that are changing going forward in version 4 which is out in beta right now is you know when you're sending transactions to to an address you can also send transactions to an through name name service address so this will be a bit more obvious if you have a typo in a name you know of course they're still fishing in tax and things like that but it's certainly a lot easier for a human to spot a letter swap in addition checksum addresses are required everywhere in version 4 of web 3 so that means that you should be looking for you know other services that return addresses and checks on Furman so we've got a new estimation algorithm coming out it's inspired and courtesy of eath gas station info and rewritten into Python integrated with web pretty you know currently you it's it does quite a bit of work so there's a trade-off there of how quickly do you need to send this out or do this estimation you know because it might take you you know a number of seconds ten seconds or something to to run an estimation because it hits a whole bunch of recent data to to do it and it does it all on the fly right now so I'll skip over the contracts and we'll whenever you interact with the contract that's got a type like string or bytes it's gonna give you back that natural native type in Python so that means you know if the contract returns a string you get a Python store if it returns bytes you get Python bites it's worth spending some time if you're not familiar with encoding back and forth between string and bytes this is absolutely critical to to understanding and working with any raw binary data is understanding is knowing ahead of time what encoding it was in so all these things are in version 4 they're coming out soon ish so there's a lot more coming as well but not enough time to cover it so node interface definitely has some things to work on coming up soon in what 3pi so let's talk about non hosted keys so some quick definitions at least how I'm using them today so local keys versus hosted keys hosted keys are kind of the traditional experience of you know starting up your own node and generating an account a local key you know for people who haven't dived into this essentially is really just 32 byte of random data so you generate your own 32 bytes of data and keep it somewhere very very safe and so if you have a local key if it's stored in your app here it's a local key if it's stored on your note it's a hosted key which also brings us to the importance between a sort of a local note or in a hosted node and by hosted I mean someone else is controlling it so like an in fury' versus a local node where it's completely under your control if anyone has access to your in to your node it's hosted by this definition so you know hosting key on a local node is your traditional case that I covered you know get their parity you create an account and then you can send a transaction now if you have if you're using a hosted node you can't just tell in fury' - hey go send a transaction with my account you know the details because they don't host your private key which is a good thing so you have to do all the work locally you have to assign the transactions and give them the raw sign transaction so that would be using a local key with a hosted node and you know since we're in numerating options a hosted key on a hosted node should never ever happen if someone's offering that they're scamming you or either by intentionally or by accident don't ever do this so and then a local key on a local node is kind of trivial combination of other things so what's kind of holding us back in this space of hosted of non hosted piece of local nodes a lot of web through JSON web through PI both have been focused on the use case of your node has your key so you just give you know the information about the transaction you want to send here's who I want to send it to here's the value and you know it takes care of packaging it up signing it and broadcasting it which means that web tree didn't have doesn't currently have a lot of good support for working with local keys where you would have to manually locally do those things I'm packaging up the transaction signing it and handing it off as a raw transaction so you know the the options for working with local keys mostly are my ether wallet or my crypto I don't I guess those forks recently I'm Harmonie you know favorite horse I'm betting on there so that's pretty good for manual tasks but obviously doesn't help too much with programmatic ones especially in Python so it's very difficult to do things like work with local keys to broadcast transaction you sign locally it's difficult to work with state channels it's difficult to do kind of authentication that you might want to do you know such as maybe your web server wants to authenticate some using someone's etherium account so you give them a message to sign they send it back to you signed and you can verify it in your Python server like Django and say okay I know you control this account without having to actually submit something on on the blockchain so all those things are a little tricky to do right now but upcoming solution in beta is a package called eath account this is a library that is essentially built to do all of these things signing and verifying transactions and messages you know trying to be as minimal an api as as we can support and still be useful this is ready for you know it's it's essentially API stable right now pending security audit so not ready for production use but that's scheduled for this month so we're optimistic it'll come out soon this will also be wrapped into what 3pi alright so a verifying contract source in a lot of ways this is the the promise of etherion the quintessential promise of it and in my mind of being able to know that you're working that the code that's running is the source that you read right it's it's kind of like the next step of open source we're in desktop open source you could download source and compile it yourself and you know exactly what's running you can't do that well with hosted services someone someone just promises they're running a open source server you can't really know that for sure so aetherium kind of enables that right it gives you the ability to see the source that's running among everyone on the network but in order to do that you have to actually be able to say okay I read this source I personally verified it is that actually what's in the blockchain and that's not trivial especially right now because unfortunately the best answer we have right now is go look for the this check mark on either scan to decide if it's really validated which is terrible this should not be the state that we are targeting I mean it's great that either scan offers the service because it's the best thing we have but we shouldn't the whole point is we shouldn't have to rely on third party trusted you know sources for these things so why is it why is it that you know we're in that state right now there's a few things that are difficult or that are required to actually verify this essentially you have to compile it yourself and then verify the output but because compilers are finicky you have to know every single detail that went into it right in addition to the obvious things like the source files you have to know the version of the compiler that was used the flags you know whether it was optimized and whatever compiler flags you might have on you have to know which libraries it got linked to and of course the deployed address so either skins sort of does both all of this and then actually does the compilation and tells you it was compiled in the same way but we should really have some other mechanism available and so this is very much in the planning stages there's a spec out that's sort of under a view and people are thinking about it but haven't gotten a whole lot further there's something called eath PM spec and it basically defines a lock file that includes all of these things and some more so I you know it's just a little clip of it here includes things like the compiler version and flags so this locked file the idea is that it includes all that information for verification as well as links to the other libraries that it depends on in the form of content address linked to the lock file of that library so you get kind of recursive lock files all the way down and and you post it up on IP FS or whatever content address store that you want and and you can verify first of all its you know once it's once its broadcast you know it's the same thing because it's you know fixed data and and you can then use all that information to locally verify depository bit okay so each p.m. spec is sort of the eventual upcoming answer to to that issue and then the last thing I want to talk about today is the all Python node so right now the situation is that pi-theorem is the stable Python node that's out that's available the prototypes that the research group works on whether it's you know sharding a proof of stake often end up well they're they're all almost all in python and often end up in in the pi aetherium codebase and additionally you know the the network is generally more resistant to things like denial of service attacks and consensus bugs when there are more more different kinds of nodes available so where are we really with that well pi-theorem unfortunately is practically incapable of keeping up with main net for a variety of reasons including everything running on one thread with the networking code and the EVM execution and other things which means that there's this sort of gap between the work that the research team does on top of pi theorem and actually getting into production it has to all be re-implemented somewhere else in other nodes of course their implementation has to happen anyway but it'd be nice if we got to at least reuse the the research teams work directly and then you know ultimately there's only two dominant nodes last I checked compared to your like dominating the network now two is way better than one we've already had scenarios where one or the other gets knocked out by denial of service or consensus issues well mostly denial of service and and so you know the other one was keeping it moving but it would be nice if we never got knocked down to even one so the upcoming answer to this is Trinity which is a brand new all Python node shouldn't say new exactly it's under active development hasn't even been released as an alpha yet this is not an official logo just a fun image I found online so the the goal for training is to be a viable maintenance of Gotham party which is of course a lofty goal so it'll it'll take a while to get there but you know right now it is it is under production and or yeah under development and the the research team is already starting to work on top of it it's a you know runs some local tests some EVM local tests effectively passing them and it's currently located so with the project called PI EVM which will ultimately be more like the EVM library that any other project could import and then Trinity will be the node that you can you know start up at the command line that takes care of all the networking and everything else that's involved with running a full node so that's definitely worth keeping an eye on as as another alternative even if people are just kind of interested in reading through how does the EVM work this is a great reference to to how execution actually carries out so just quickly a little bit of news the web 3pi version for launch is scheduled for at the end of the month and mostly just because I've got a continued European vacation going on last part of this trip so B should be end of the month everything going well and so the things that I mentioned should be out quite soon actually if you're if you can't wait then you can you can't install it on pip already with with pip pre so for anyone interested I've also got a workshop tomorrow it's sort of the idea is kind of a beginner friendly interactive workshop where you can bring your own ideas or use one of ours to get started you know walk around see what people are interested in doing and just kind of turn into an office hours help people out wherever you get stuck and and just hack you know no point in coming if you don't bring something to hack on that's that's the kind of workshop it'll be so just to wrap up you know how do we actually overcome gravity you know it's not gonna be by pure marketing it's not going to be by like buying and selling tokens or buying and holding tokens the way that we actually defeat gravity is by building things so I hope hope you'll join contribute however you're interested even you know tell us what you think is could be improved about it everything helps and with that I'll say thank you and [Applause] anything any questions yeah so github yeah sure so the question was where do we find the the trinity project so it's on github the etherium org so github.com slash etherium and then right now it's co-located with the PI AVM project so github aetherium slash PI - AVM the HBM package how does it fine for a consensus or Authority how how do you deal with different source code that can compile to the same so this is purely the spec as it is right now so the question was oh yeah I guess you got the question on Mike so the the spec is purely a matter of defining a file format right so it's just saying you know it includes the compiler version and everything there's no you know once you have all that information you don't oh well so it so in addition to things like the compiler information it also tells you which network it was deployed on right so so once you have all that information there's no there's no more trust involved you can do everything you want to do locally because you can get the code out of that network address and then use the sources that are linked in the lock file and use the options that are linked in the lock file and do everything locally so there's no sort of consensus or Authority left there I mean I may have must misunderstood I'm sorry okay so given that you have an address of a contract how do you get the source code so in Webb 3 there's a method called get code and this is in all the Webb threes and so it just takes an address and it gives you back the bike code at that address so this is sort of a built in part of the etherium network is you can directly access the bike code the source the source part is you're still gonna have to find that somewhere right you're gonna have to find like whoever is offering whoever posted the contract is gonna have to make the source available so it's still this doesn't prevent people from never releasing the source but but once they do release the source they can they can simply link to a lock file and that's all you need as opposed to you know some long list of things that you need to develop it thank you very much sorry I I yeah you next speaker [Applause]