Better Smart Contracts with Behavior-Driven Development with Thomas Vieira and Neodaoist

so uh we're going to give a talk today on better smart contracts with behavior driven development my name is Neil Dallas yeah come on in guys grab a seat we're just getting started hey how's it going uh my name is Neo daust I'm a software Dev solidity Dev product person uh working on a D5 protocol right now and I'm petrotech on Twitter you can call me Pete I've been uh working on software for about 20 years web one web two now web 3 and uh using Behavior driven design and in many projects across many different areas okay let's get started here how many of you principally write code that's what you do a lot okay how many of you always are almost always write tests with your code a few okay now you were at a we're at a ethereum conference here and you're at a talk on testing so put your hand down if you're the only one on your team that writes code or that writes tests with your code so this is the problem this is the problem we're trying to solve get everyone on our team everyone who's writing code to write quality tests um some agenda what we're going to talk about we're going to talk about the what and why of bdd transition into some bdd mechanics and then we'll talk about bdd as a tool within your your larger toolkit nice so starting off with what is uh bdd so Behavior driven development ultimately what we're going to have is a plain English executable specification that describes how our system in this case our smart contract or our solidity code should work it's going to evolve alongside our design and implementation and our documentation and so we'll see some examples in a little bit one thing you might hear in this kind of bdd world is it's a token for a conversation this isn't some type of like product requirements document PRD spec that we're trying to write that can like save us from having to talk with our users and our stakeholders this is a token for a conversation it's kind of a placeholder and it will help us remember what we've talked about and really describe it in an unambiguous way so it ties all behavior that we're going to write in our smart contracts or our broader system to a user and a purpose so we know why we're writing any piece of code um I kind of mentioned this it's going to help us express complex functionality unambiguously but in a like human readable friendly approachable way and finally we think this has a lot of value particularly in an environment like an adversarial environment like the evm and when we're doing immutable one-way deploys a little bit more akin to some of the properties of Hardware development and that's where we think this uh kind of rigorous way of writing a spec will be really helpful and we're going to share some of what we've done uh with like half a dozen projects or so so far looks in the real world well then introduce a few terms here first of all bdd what are we talking about here Behavior driven development is this is a framework for specifying a software system that ends up being executable code that you can understand so it's user stories illustrated by examples cucumber is a framework that implements bdd it was originally developed by Dan North quite a long time ago now implements bdd by executable feature and scenario specifications it was originally developed in the Ruby world but it's imported to many languages including Java but more appropriate for smart contract development JavaScript Python and rust gherkin is the language of cucumber and we're going to be seeing some examples of that in just a second it is specific enough to be executable and to be useful to a developer but it's natural enough to be understandable by non-technical people that's what makes it a token for a conversation and you'll see how that works in just a second so we have features scenarios and step definitions a feature is a desired product functionality that we're going to illustrate with scenarios it always involves a user who's trying to do a specific thing it's a sort of like a user story but a little more specific a scenario is an illustration of a feature it takes you through one particular path as we try to understand what a feature looks like and it uses formal steps and examples and then a step is a definition using this given when then pattern that's become quite common in many test environments that illustrates the scenario so vdd is important even if you're in the habit of writing unit tests and you write great unit tests we need to go a little bit deeper for an application here we have an example where we have two unit tests this Outlet works fine the towel bar works fine they don't work together in a way that satisfies a user so we need to integrate our system together and test the whole before we can be confident to release it to end users let's take through an example very simple example of minting at nft so this is a project called Spork start something that we're actually working on for eth Denver here and I'm not going to describe it for you you'll see it in this code here so as an attendee of East Denver this is the who all features ultimately come back to a human even if something is quote unquote system there's always a human at the end of the pipeline somewhere maybe it's an administrator maybe it's the contract owner try to figure out who the human being is that we're trying to do something for here why in order to have a Memento like my experience and people I met what I want to men a sport star now we have a framework in Mind of a feature we're trying to build why we're trying to build it and who we're trying to build it for so here's a happy path minting a sports star using usdc so now we're going to get into the given one then pattern so Givens are the situation in our system smart contracts or state machines it's a state machine before an event takes place so given I have five usdc in my wallet when I meant a sporkstar then I should have a sports star in my wallet and I should have zero usdc in my wallet this is not super complicated but it illustrates the idea of how this works let's look at a contract that many people are familiar with wrapped teeth if we were writing the specification for rap teeth it might look like this feature rapid unwrap eath as a web 3 user about to interact with adapt in order to use Ethan context that require an erc20 token I want to wrap Ethan an arc token and then I want to redeem that back to eth deposit what's this look like given Alice has 10 Heath and Alice has zero wrapped teeth and the total supply of wrapped teeth is 50. when Alice deposits to Ethan to the contract then Alice should have eight eth and Alice should have two rap teeth and the total Supply should be 52. so I'm going to pause here and ask how many people here describe themselves as non-technical not Developers well we should have had some developers come to this talk but those that are non-technical keep your hand up if if you kind of understand what's going on here and that's the point of this right this is specific enough that if you're a developer you can make this work turn it into a real world test but if you're having a conversation with someone who is not technical they can also understand what's going on and that's that's the beauty of this particular gherkin language nice thanks Pete so a little bit about how much to automate and we were kind of thinking about this about web 2 and web 3 projects that we've worked on there's a few different levels to this so level zero it's like just write the feature file have the conversations do the thinking you know often you'll be writing a feature file you can revise it we're going to talk about how words and pictures fit together a little bit later but it's basically just writing your spec in such a way together and co-creating it with your users or your teammates or your stakeholders so that would be great right there that would be level zero level one would be basically write your feature file and then write so where Foundry Forge fans write Forge says and we're going to see a sneak peek of how GitHub co-pilot and other AI tools can help this um this isn't like the most driest way to do it right because if I have a feature file and then I have a separate set of like a Forge test Suite when I make a change I'm now having to update it in two places but it's a pretty good way to structure your tests to be sure uh level two would be fully automated cucumber step definitions with like uh we've done cucumber RS plus ethers RS and so we'll see an example of that uh you could do cucumberjs plus ethers JS or hopefully soon we could do cucumber solidity or some such thing where we have cucumber feature files that are being driven by solidity code itself so we can Define our steps in solidity and we don't have to contact switch to another language so yeah let's see an example of like what level one could look like where we're writing this feature file and some Forge tests and then we'll see level two also where it's fully automated with ether's RS okay so now let's imagine we're in an IDE this is in our Forge test file I've cut some of the boilerplate some import statements and things like that out so that you can actually see it on the screen we're going to go through the same scenario that we just saw minting a sports star using usdc and actually implement this and turn it into a real test so we have a function this just looks like any other unit test framework test mint sports star for usdc and we're going to start implementing this now at this point for this level one type implementation we're not using a robust cucumber framework we're just using Forge so what we're literally going to do is copy and paste our given when then specification into this Forge file and start and start implementing so our first given was given Alice has five usdc in her wallet now we think all we have to do is think about the specific step what does that mean how do I make sure that Alice has five usdc in her wallet well in Forge we would start crank so that's a we're going to act as this address and we're going to use this utility function write token balance to put five usdc and Alice's wallet I should mention at this point that we're running in a a local test environment a fork of mainnet but this is happening locally which is also why we're ignoring gas fees and other such things when Alice mints a sports star what does that look like well we have to approve on the usdc contract and then we call this method on our contract meant for usdc passing in the token ID that we're minting our address that we're minting to and the amount in this case this is also I should mention it's an ERC 1155 type token so we're mentioning one of token id1 then Alice should have a spork star in her wallet what's that look like that looks like an assertion we assert that our has token method with Alice's address returns returns true for token ID number one I should also mention at this point that GitHub co-pilot turns out to be really good at this by the time you paste these comments in here you're given one then about 80 percent of the actual test code can just be generated by by AI it's a it's a huge Time Saver in writing test code okay last step again something that GitHub pilot generates just fine we assert that Alice now has zero usdc in her wallet so now we've gone from the specification to a unit test that we can actually run we can actually put in our CI Pipeline and now we're asserting that our smart contract behaves the way we agreed with our user that it should sorry last step is a payout address should have five usdc where did that uscc go it went to our uh you know our payout address now we're going to take a look at a slightly more complex example um this is a project we did that has a random reveal mechanic so a series of nfts that uh reveal their particular tier of Rarity after after some time so here we're using a little bit more advanced syntax of cucumber but the same idea we're going to focus in on when we reveal the art then there should be 222 tokens with the following metadata and quantities the key thing here well really we have tier descriptions we have the Rarity these are attributes that that end up being said in the nft in this case the quantity is the key thing here so we reviewed with our client what are the quantities you want to have in the case of a sellout they came up with these numbers let's make sure this actually works and then also calling reveal a second time should not change any tiers this is immutable once reveal this happens so in this case we used cucumber RS to implement this in Rust code first we have to set up a few things we're using the again you could if you don't like rust you can use cucumber JS and do the same thing in JavaScript or cucumber in Python if you're writing Viper code you can do the same thing in many different languages this happens to be rust so we have to do some setup here we have to set up our Fork environment and then this ABI gen macro is generating rust bindings for our smart contract that we can then interact with in our rust code so we're setting up our Fork network setting the chain ID and uh and running the rest of the scenario this is a step def so this uses a macro that matches our when pattern so when and this matches the we reveal the art this tells the Cucumber framework this function matches this step def so when it's running through our our feature file and it sees we reveal the art it's going to look for some function that's annotated with a regular expression that matches that step here it finds this one it's an exact match so it's going to call reveal art it's going to pass in this world object which is something that the Cucumber framework maintains you can use this to store State between your step definitions so in this case we've deployed our contract in the setup step and we've stored that that contract reference in the world object cucumber passes this into each step def so that you have this context as you go through the steps of your scenario so in this case we're going to call the reveal method on our contract now this is where you see why we're doing this in a rich language like rust rather than trying to do this in solidity so this is matching again a regex the distribution should be so many ultra rare So Many rare So Many uncommons Etc and what cucumber does is it passes our our world object in and then it passes in each pattern match of this regular expression as parameters into our method so we get the number of ultra rares the number of rares Etc and what we're going to do is we're going to call our contract method we're going to use a range operator here luckily this is a local test Network so we're going to call our contract method 222 times but it executes extremely quickly we're going to check the tier info for each one and sort it out and check that it matched our expectations another useful feature of cucumber is the idea of a scenario template so in this case we had this conversation what happens if we don't sell out what happens if our supply isn't 222. what if we only sell 50. what if we only sell 100 we had this conversation with our client and we agreed how the distribution should work for any particular number of sales one of the things that happens here is we always have one in each tier now what cucumber does in this case is it's actually going to execute this scenario one time for each row in this examples table so we're actually going to run the scenario six times in this case passing in these parameters the nice thing about cucumber is you can actually reuse step definitions so you see the when we reveal the art and the then methods are the exact same thing we did in the previous example I already wrote the step definitions for that so I don't need to write them again cucumber will find that and it'll pass in these parameters and now we've done a very robust test again local forking we're forking we're deploying six times we're revealing and we're checking and this still executes in a couple of seconds nice uh okay so we're talking about bdd that probably brings to mind tdd do you really have to write your test first um well I don't know how many people here do write tests first at least some of the time all right some hands some some yeah yeah yeah yeah some babies so I would say like no but don't get too far ahead of yourself right bugs live in untested code the more source that you write untested uh if and maybe inevitably when you run into an issue it's more to untangle either to like debug or make the code testable um so don't get too far ahead of yourself and you know where does confidence come from so when you're refactoring under green like under test that pass you know you change some of the code and all of a sudden you have a failing test then you start writing tests and then you start adding the new functionality that's a really good feeling it's like it's a great feeling um the other day working on a contract I changed from like kind of a naive like one step ownership change to like a two-step pattern where you you know nominate a new owner and then you accept it I made that change like a tdd style it felt really good because like I had a set of like happy path tests that had to change or like one or two and then some sad path tests for all the ways that it could go wrong felt really good that now I had tests that asserted that behavior and that I was as confident as I could be I think that I implemented that feature correctly just to put a finer point on that though you should write your test first but sometimes we don't sometimes we're in the code we've really written a lot of tests and now we find out oh I needed an only the only owner attribute on that method you can put the attribute in and then immediately go put your test in that type of thing is okay that's what we mean by don't get too far ahead of yourself what you don't want to do is have a hundred lines of code and now you go to write a test for it and you can't even remember what code you've written something breaks and now you're not sure what broke because you've changed six or seven different things that's the point totally um and yeah afterwards too I would be curious to hear people's experience both writing tests first writing tests later hopefully not writing tests never but uh not every not every software problem probably requires it but I think there's a lot of benefit and that's kind of what we're trying to espouse today okay so this is not a silver bullet like right this is a tool in your tool kit and we're going to show some uh show how it fits in our tool kit because as solidity devs or evm devs uh there's a lot of different things that we need to do to ensure that the smart contracts that we're writing particularly if they're going to be holding user funds are as airtight as we can so that we can sleep well at night so that the Auditors are happy with what we present to them so they can audit our code the best that they can and ultimately we you know solve a user's problem at the end of the day uh so fitting in with your CI stack one thing is you know if we think of kind of a testing pyramid at the bottom unit tests probably more of them but they run a lot faster as we go up we might start forking mainnet we might start integrating more with actual contracts that are deployed it's pretty nice so you can see in this project we've got some GitHub actions like unit tests then the Cucumber tests then uh code coverage type stuff it's pretty nice because we can see with a well-named scenario if we were to break something hey what broke what part and we can see that if the build fails so it fits pretty nicely with your CI test to be sure having these as different jobs is kind of nice too because the longer running ones like uh maybe not slay there but something that takes like a really long time to run you can have later and so you can fail the build early uh if there is actually something wrong with it uh so okay so one thing that I maybe wish that some of the bdd community uh if you will would do focus more on is pictures you know we we know the saying pictures tell a thousand words there's something that a picture can really Express and communicate and help build shared understanding between me the users and stakeholders that just straight words or prose like we've seen with these feature files won't do so a user story map great way to express this functionality if we want to really get more specific with how State changes maybe we could use a sequence diagram some type of a process flow diagram maybe UI comps if there's a big user interface element it dovetails really nicely we found with these bdd feature files uh and very interesting shout out to horsefax for this great repo where it talks about weth and variant testing but this can fit really well and we'll actually have a a feature request I guess for cucumber to be actually defined in variants inside the feature file which I'll show a sneak peek of threat modeling shout out to zelik who produced this threat model for valorum a protocol that I'm working on but the idea of really explicitly saying what every function does the happy and sad paths and then the controllable inputs and all that type of stuff it can fit really nicely with the unambiguous nature of this cucumber specification that we're writing okay and then finally just putting my product manager hat on for a moment uh I don't want this to come across we don't want this to come across that you can sit with a user ask them what they want you to build write it up in cucumber write a smart contract write adapt ship it and then you've solved their problems right like at the risk of sounding cliche we've probably heard that Henry Ford quote if he asked users what they wanted they would have said Faster Horses users are experts in the problem not in the solution it's our job as developers and Builders to be aware of the technology that's just now becoming possible so that we can solve the problems that users come to us in novel ways that is a better solution for them so I would be remiss if I didn't mention this at the end that it's a continuous Discovery process of user interviewing rapid prototype testing and other things that help us get to the confidence level that we're actually going to be solving their problem and cucumber can be a great way to actually articulate what we're ready to build particularly where we're ready to build when it's going to be a one-way immutable hard software deploy real quick on that point sometimes when you're writing a feature file you'll it forces you to think through a problem and you realize you don't actually know how the behavior should be you're getting ready to talk to your stakeholders in that case put a question mark given something when something then I don't know question and then when you're reviewing that with your stakeholders it prompts you to talk about that you say what what should happen here let's let's talk about it let's get it captured now you captured in your feature file and now you remember that you talked about it nice Okay cool so some benefits and costs to automation because we're talking about probably maybe for some of us doing things and spending time that we aren't doing today so there's a cost to that uh let's talk benefits um better CI we think it can be valuable fitting into your CI pipeline uh step reuse so I'm sure there's like unit testing patterns that we use where we you know have whether it's composition or inheritance type stuff to save common things that we're interacting with our contracts and our tests um like you showed Pete some things were on richer expression both in plain English and then verification with some nice data structures you can refactor with confidence you know there's nothing that would would stop you from writing this kind of given when then Style just in your unit tests honestly that's super valuable I've done that for some projects that don't have a lot of like end-to-end feature testing requirements that's great and then catch bugs and design flaws early uh hopefully before sending to the Auditors or before shipping um okay but is this for free well no there is some initial upfront cost you have to set this up there's some ongoing maintenance costs you have to maintain your test and uh currently it requires context switching to a language with Native cucumber support like JavaScript or rust where we're not writing our smart contract however in every project I've ever used Behavior driven design in and I've been doing this for at least 10 years those costs have always been amortized by the time we ship for the first time in other words it actually over the course of getting your product out in the marketplace solving problems for real people it accelerates nice uh so a couple uh or a few wish lists that we have here as we're winding down our talk today so connecting to Foundry this would be amazing if we could write our step definitions in solidity so we can drive automation basically interpret the Cucumber file run Forge tests and then we don't have to contact switch like except for like plain English and solidity that would be amazing let's make it happen I know there's some I know there's some some Rust Giga brains in the audience that can that can help make this a reality uh invariant hooks so this would be an idea where we could write our invariance in the Cucumber file and get a little bit more numerate with like some math and things that we're trying to keep as an invariant and then run that before and after every step in the scenarios that we have that would be amazing and then this last one you may have seen that scenario online I think we showed a couple it's like quasi-fuzzing but if we could have fuzzable scenario outlines where we could exercise boundary conditions and really throw a lot of different random data in random inputs and I think it would let us express again particularly in evm smart contract World D5 where we're there's a lot of math and numbers in it we could then fuzz from our cucumber files which could be a great way to express it so I think we have one more side quality is a first class citizen let's treat it as such so thank you so uh yeah so I'm going to leave you all with a challenge I went to a conference maybe 10 years ago on on cucumber and the speaker gave this challenge I'm going to pass it on try this the next project you do just take on faith that this works try it for one project if you don't like it if you hate it don't do it again try it end to end on one project see if you like it I took that challenge and I will never do a project without bdd again yeah and uh connect with us on Twitter and yeah any questions comments or ideas I think we have a few minutes left thanks uh another uh bdd jargon term but as you're building these projects are you doing something like outside in red green refactor with your uh with your tests like can you describe just a little bit how it benefited you in the relationship between your integration tests and your unit tests uh yeah totally so the question is about the kind of outside in red green refactor Loop and how that can work for integration in unit tests uh well I'll give a good example so in the hackathon I'm working on a I guess we can I guess we can leak this working on a a game we'll just say it's a game and when I'm sitting down to write the source code what I did initially was write some given when then features to say okay so this is the scenario that I'm trying to do then I went over and wrote a failing unit test wrote some solidity code wrote another failing unit test for the revert conditions uh you know expect revert in Forge wrote the solidity code with you know the the custom errors and did this a few times then I could zoom back out so we actually had a good graphic and another presentation we've done of this where there's two Loops there's a big bdd feature Loop where we write a failing feature test then there's a smaller TDE Loop where we write a failing unit test make it pass refactor write a unit test make it pass refactor and then we finally zoom out until we've implemented enough functionality for the feature then we write our next failing scenario we go in we do the red green refactor Loop for the unit test and then we Zoom back out and so just doing this locally and then you can start to see when you open a PR and you have some failing or you have some passing feature tests and some passing unit tests it's really kind of building the spec alongside the tests alongside the implementation so it's been quite valuable any more questions all right yeah come find us afterwards if you have ideas why this could work why this wouldn't work why would be valuable why be wasted time would love to hear more so thanks again everybody for coming out and have a great conference have a great foreign