SE Radio 562: Bastian Gruber on Rust Web Development

Transcript brought to you by IEEE Software magazine.
This transcript was automatically generated. To suggest improvements in the text, please contact content@computer.org and include the episode number and URL.

Philip Winston 00:00:16 Welcome to Software Engineering Radio. This is Philip Winston with Bastian Gruber. Bastian has more than 10 years of experience developing software, much of it as a freelancer. He was part of the official Rust Async working group and founded the Rust Intel Berlin Meetup group. Today we will discuss his new book, Rust Web Development. Thanks for joining us Bastian.

Bastian Gruber 00:00:40 Yeah, thank you for having me.

Philip Winston 00:00:42 I’ll just say a few words about Rust to set the context. The Rust Programming languages has roots going back to 2006 and the 1.0 release was in 2015. Rust is a systems programming language and is considered a safe alternative to C and C++ without compromising performance. The Stack Overflow developer survey has named Rust the most love programming language every year since 2016. An example usage is AWS uses Rust for its Firecracker system, which powers AWS Lambda, its function as a service offering. And in 2022, Rust became the third programming language used in the Linux kernel after assembly language and C. So, Bastian and there are many popular languages and frameworks for web programming. Why do we need another one?

Bastian Gruber 00:01:35 That’s an excellent question and I get asked this a lot. In my experience, Rust is the first real different language in the other languages you could say you move from Python or to Noches(?), you move even to GO, but they sort of have the same feel and there’s nothing really radical new with Rust you have the type system and the compiler and I think that you also have a whole group of people who come from a Ruby and Noches(?) which helped design the language. So, it feels a new, a new beginning. It feels a very new and fun language to work with and it’s not just a new syntax, it’s a new thinking, new compiler. Yeah, it’s new. It’s radically new.

Philip Winston 00:02:29 Okay, so you mentioned a few, but what does Rust bring to the table relative to web programming that you feel makes it a viable contender?

Bastian Gruber 00:02:38 I think you first have to step back and say, why would I write it in Rust? Because I think the crux about it is, it’s not really batteries included when it comes to web programming or to create web services, but a few, but it has enough to get started. So, I think when people try to approach it, they look at which packages can I use if I come from Python or from GO, how does it look in Rust? And I think that’s the wrong approach. I think it helps a lot to look at a type system and the compiler and what this means for the longevity of a software project. It makes it more robust and easier to work with. And then you can think about the Rust compiler makes sure that your black memory is threat safe without a runtime, which both helps in writing applications for the web because the as in Kronos, the way of writing code is still quite hard sometimes and Rust makes it safe. And then after you’ve compiled your application or your web server, it runs very efficient and with not a lot of resources required. So, it’s both safe to write, to maintain, but then in production you save costs and you can be sort of sure it’s safe throughout its lifetime.

Philip Winston 00:04:15 So I can imagine two paths to using Rust for web programming. One is learning Rust for other applications, maybe getting good at the language and then deciding to use it for web programming. And the other would be to jump right in and learn Rust and web programming at the same time. And that might, your book might bear on that path, but can you comment on those two paths, how viable each of them is?

Bastian Gruber 00:04:40 So the thing with Rust for all, for all its advantages, you need a sort of a base. The base is not a lot, it can take a month or two, three months maybe read the first 4, 5, 6 chapters of the Rust book. So, if you want to jump straight into web programming, you will see that it will struggle more than if you would pick another framework or like language. And this is you have to get used to how the Rust compiler thinks and wants to guide you. So, if you jump straight in, you will get frustrated. But if your mindset is I have to learn part of the language at least to know why I have to write a certain way, then you will have more success. If super used to maybe C++ if you’re used to three, four different languages you might succeed. But if you just have one step in your past, I would recommend to learn the language first, write ACLI application first and then jump into web programming.

Philip Winston 00:05:48 And the Rust book I think you mentioned is the Rust Programming Language by Steve Clark and Carol Nichols.

Bastian Gruber 00:05:56 Right. You can purchase a book but it’s also open-source and free to read in the browser.

Philip Winston 00:06:03 Oh great.

Bastian Gruber 00:06:03 Sort of a standard reference on Rust.

Philip Winston 00:06:08 So are there web programming environments or situations or applications where you think Rust is not the right choice for web?

Bastian Gruber 00:06:15 It’s really hard because there might be something I’ve never touched or heard of. If I think back in my career now and I think back, okay I used here Node, I used Java, I used partly Go, I don’t see a reason why I wouldn’t use Rust for these use cases. It can be that you need to prototype really fast, you need on Monday and you know by Friday you need something working to show to an investor or to a customer and it just has to show a few things. Then Rust, if you’re not as experienced might not be the right choice because the compiler wants to make sure your program runs safe and if you prototype you know the pitfalls and you don’t care if it’s safe since this application will never see the light of production. So, if you really want to prototype fast and are not two, three years into the language, it’s not the right choice for you.

Philip Winston 00:07:14 I can see that, I can’t remember if you covered this in the book, but how did you personally start using Rust and then how did you personally get involved in Rust for web programming?

Bastian Gruber 00:07:25 So I was on my, I was a contractor, I had a Nocheís(?) project and around Corner was in Berlin there was the Mozilla headquarters in Berlin and they had a meetup called the Rust programming language. And then I googled and thought hey this sounds interesting, it seems assistance programming language. And I was sort of done with Nocheís(?) because I’ve done 10, 15 different projects, I saw all the pain, I didn’t see how Note can solve it so I thought I need a new language to solve these problems. And back then it was prior Rust 1.0 so it was I think 2014. So, I’ve tried it, I thought I don’t understand anything about it, I dropped it again and then in I think in 2017 or 18 there was a JavaScript conference in Barcelona and Steve Clubnick, he presented the language in a little booth and he also said there’s the first Rust meetup in Barcelona, feel free to join.

Bastian Gruber 00:08:25 So I joined there, I saw the language and what fascinated me was the people who joined there were open-minded. They were open to fail, it was not as strict. I joined a few go meetups, and I felt a bit out of place and with Rust I just felt home personally so I thought hey that’s great. I want to also start a meetup in Berlin about this where people can share their thoughts about the language and this is where I thought I first want to start a meetup; see which people this language attracts and maybe this is something for me. And after starting the meetup I thought yes, I think I want to pick up the language. So, I wrote a few blog posts, I was on paternity leave back then and I saw I want to move from Node to Rust so I dig deeper into how to write a web service in Rust. So, I blocked a lot and then I cut my first Rust chaps and from then I’m in love with the language.

Philip Winston 00:09:21 Great. Okay, we’re going to go through a few features of Rust. This is very much a subset that are covered in your book and even a subset of that. But we want to give a flavor for what Rust specific features and how it impacts web programming. I want to refer to two previous episodes for more details on Rust itself. There was episode 279 with Florian Gilcher on Rust and I guess he’s a core Rust programmer and there was episode 490 Tim McNamara on Rust, the second one was 2021, the first one I think was 2017 and we already mentioned the Rust book. So, here’s a few features I just wanted to kind of get your read on. So one is, is one of the hallmark features which is performance. What enables Rust to be as performant as the fastest languages?

Bastian Gruber 00:10:16 In my read I’m not a compiler engineer but the compiler like what transforms your Rust code to a binary and machine code is the compiler, there is no runtime attached. So, once your code is compiled and in an asynchronous way, you can think about a state machine, your asynchronous functions will be transformed into a state machine and the Rust compiler make sure that the memory you used and you want to write to is always safe and threat safe. So, after you’ve done the compilation process there’s no overhead attached in Java, in no chairs ingo(?). So, this makes it very efficient during the runtime because it doesn’t need this garbage or this active garbage collection.

Philip Winston 00:11:19 Yeah. So, one of the other features I was going to ask about is ownership, which I think it relates to the lack of garbage collection. Ownership is not a term I’ve heard in other languages but can you talk a little bit about how that impacts your sort of day-to-day thinking with the language?

Bastian Gruber 00:11:38 This is a topic many new Rust developers struggle with and I think even if you go on and do more complex applications, this will have an impact on how you write your code and how to think about the code. And this is a caveat of Rust but also its greatest feature. It basically says that not more than one process or active part of the application can write to a memory space. So, you can have multiple reads but just one write. So, if you pass a variable by value to another function, this functional owns this space in a memory and if you want to pass it multiple times to different functions, you have to think about how would I do this so I can make sure that I adhere to this ownership principle that just one function can actually manipulate the memory. So, it’s, I’m thinking about this is where the performance comes in and the no active runtime comes in because the compiler makes sure that not two processes at the same kind can manipulate the space in memory but therefore you have to think a little bit about it. The compiler is going to help you and show you how to do it in a different way. But this has an impact on your architecture of your application.

Philip Winston 00:13:07 I think we alluded to it but let’s talk directly about safety a little bit. I think people that maybe have never programmed in CRC++ don’t fully understand just how unsafe those languages can be and I think it relates to what do we call a crash in the language and in C and C++ a crash can be the entire application goes away or it does something unexpected, it writes to memory which gets written to disc and there’s garbage. It really can be sort of a catastrophic crash wherein safer languages you expect things can still go wrong but maybe there’ll be an exception that you could handle or maybe Rust has its own term I guess. So, can you talk just a little bit about safety and what that means?

Bastian Gruber 00:13:58 So in Rust it’s not possible to have sort of a no pointer. You always have your return like a result type or an option type which can either be true, false or there is something or there’s nothing. And if your function is returning, let’s say you want to write or you want to open a file and this function returns the status of this operation, you can return a result which says yes this was successful or no I couldn’t actually write to this file. And then the other parts of the applications who call these functions have to always deal with these both cases. So, it’s not possible and there’s a caveat you can overly say I always just cover the success part and don’t really handle the not success part nicely. And then you have a runtime problem but it will never write into a memory space you’re not allowed to write. So, the architecture of the types is made so you can always handle the not successful case and the Rust compiler itself makes sure that you don’t write in any other user space memory from other applications. So, these are the safety rails which Rust provides.

Philip Winston 00:15:33 Let’s talk a little about the type system and then we’ll get into tooling and then frameworks. So, okay the first element of the type system I wanted to talk about are structs.

Bastian Gruber 00:15:44 So structs in Rust, you can say it’s a class or an object which defines its members. So you can have a struct which is an animal or a dog and it has certain attributes like name and also so like, all itís member types. And then in Rust you take a second step and implement this struct where in this implement block of this struct you will define the functions for the struct. So, you define your objects and its members first and then if you want to attach certain functions to this object you have an implementation block which is called an Impo (?) and then the struct name. And then you have for example, a new function wherein Rust, there’s no such concept a constructor but you can define a function called enum(?) and once you call it you return an instantiated type of this struct back to the collar (?).

Philip Winston 00:16:55 How about generics? The syntax looks a little bit or a lot like Java and C#. How are generics used in Rust?

Bastian Gruber 00:17:05 So the power of generics is that you, a term called trait bounding where you can say you don’t really know which actual type you pass through this function. So instead of saying this is an end or this is a string, you can say this is A or B or C and then you have a wear block where you can say this type, I don’t know what it actually is, but it has to implement certain behavior. So, if you say, I don’t know if it’s a string or a number, but this generic has to have the ability to print its context or its contents to the console or to form it into a string. So now if you call this function and you say I pass whatever, but I must make sure to implement this one function to this type to be able to pass it down to the function.

Bastian Gruber 00:18:12 But it’s a powerful way of saying I’m not restricting myself to a specific type because your application can grow, but I limit what I can pass down to with certain traits and traits in Rust are behavior. Like a struct dog can implement the behavior barking and sleeping but also other maybe a cat can do the same or a giraffe. And once you write this function you don’t really say I can just pass a doc down. You say I pass a generic A and this A has to implement a function called barking and a function called sleeping and then I’m fine.

Philip Winston 00:18:56 That was going to be my last topic was traits. So that’s a good summary. So, let’s get into talking about tooling. One of the things people really seem to like about Rust is the tooling. I’m just going to go through five tools and just kind of give us a sense for what they do maybe with respect to web programming or maybe in general. We’re not going to go into detail but let’s just talk about these. So, the first one is Rust Up and I gather this installs the compiler?

Bastian Gruber 00:19:28 Yeah, so this lets you update the Rust compiler. So if a new Rust version is being released and this is done every six weeks, you’ll do Rust Up update and then you can fetch the latest Rust version and install it and you can, specify which version you want to work with. You might have a project where you lock in the version maybe from last year because you it’s not safe for you yet to upgrade to the latest or you don’t want to make sure it’s up to date. So, with also the Rust Up, it’s basically your version manager of the Rust compiler.

Philip Winston 00:20:13 Okay. And how about Cargo? I think that’s the package manager. Can you just kind of say what Cargo is used for?

Bastian Gruber 00:20:21 Yeah, so Cargo lets you create a new project, build a new project, add the dependencies to your project. So, Cargo is basically your tool you will see the most on your command line with Cargo new and then the project name you create a new project with a scaffold of a new Rust project and you can with Cargo add and then the dependency name you add a new package which updates your dependency file and adds the dependency to your project. You can also have different built steps so you can have a Cargo built and this builds a development version of your application but you can also have a Cargo built dash dash release and then this builds a smaller more, yeah smaller, more optimized binary for deployment.

Philip Winston 00:21:20 The next one is Rust-C I got it. This is the compiler. Do you ever call Rust-C directly or do you call Cargo only?

Bastian Gruber 00:21:29 I haven’t experienced yet a scenario where I would use Rust-C by hand so I always and you would also always probably just interact with Cargo. I can’t think about a case yet where you might use Rust-C directly.

Philip Winston 00:21:46 And one of the features of Rust-C are extensive air message is I remember in the book you did a nice job of showing someone make a change, it produced an error, you’d explain the error and explain how to fix the error. I thought that was really helpful. Can you explain a little bit about these error messages and sort of how they direct you towards possible fixes?

Bastian Gruber 00:22:11 So in the beginning I and we talked about was you should learn a basic set of Rust and then you have the ownership principle which can be a bit challenging to learn? But the compiler, if you make a mistake it can help you direct in what you should do instead. And this is not always the right thing it suggests but in 90% of the time and in the beginning of your journey it’s almost close to a hundred percent of the time will tell you what to do instead. So, if you don’t adhere to the ownership principle it will tell you what to do. So, you comply to the ownership principle for example. And that’s also the hard part. If you come from a scripting language, you are used to debugging, printing out stuff under console and then trying to fix them in Rust, you will always face the compiler because it makes sure that your Rust code is safe and correct.

Bastian Gruber 00:23:15 So you will have to get used to lots of compiler messages if you compile, you fix them and once it’s done you can be sure wow, now it’s finally done and I can actually run my binary. And this is a bit of a different mindset from other languages where you can run your binary or your script quite soon but it can fail and you don’t know where it fails. But the Rust compiler will tell you every aspect what you have to fix until it’s finally able to compile and this will stay with you in your Rust journey. So, the suggestion is to get used to and befriend a compiler see the compiler as your peer programming buddy, which is also nice because if you’re quite new to the language you don’t always have to go to a forum or to stick overflow. You can trust the compiler to learn from. It will teach you how to write good Rust code.

Philip Winston 00:24:14 And if I remember it’s actually showing you where in the line the error was not just . . .

Bastian Gruber 00:24:20 Oh yeah, it shows you for example, hey this variable is moved here and please add this function here so it will fix the problem for you. So, it shows your failure and then it has an info section where it suggests a better way of solving this problem.

Philip Winston 00:24:44 Great. The fourth tool I have written down is Clippy. I can’t really remember what this one does.

Bastian Gruber 00:24:50 So Clippy is one step up from the Rust compiler, Clippy is sort of right idiomatic, Rust coat. It’s a tool it was not part of Cargo in the beginning and now it’s part of it. So, there are certain rules in JavaScript it would call be linting? So, you can run it through your code and it will say, hey there’s a more Rusty version of writing this part of the code. Or you can even say I want you to notify me if I forgot to write comments for this structure for this function. So or like, hey, you didn’t return a proper result type for this function. So, it makes sure that your code is up to the highest standards and you define of how high your standards should be based on a conflict file and different type of rules you want to have.

Philip Winston 00:25:51 Okay, the last one I have listed is looks Rust format, Rust FMT. I know there’s a GO format that’s super well adhered to in the GO community. I don’t know if this is similar, how widespread is the use of Rust format

Bastian Gruber 00:26:08 I think pretty widespread. I’ve seen a few projects now where people, so then the Rust formula needs a conflict file and there is a same set of rules as maybe Clippy has but people sometimes forget to use it or don’t care like what I for example, it’s able to group your imports for a file. It sorts them alphabetically, it makes sure to group them or it can even nest them nicer. So, it formats your code to a certain set of standards which makes it easy if you work in a larger team, you don’t always have to comment on hey please write this in two lines or please reorder your imports. You just run or you like in a team make sure that you all the same type of rules, you write it in a conflict file and then in each commit you just make sure that you run it once and then it’s done. So, it’s making sure your Rust programs look neat and tidy.

Philip Winston 00:27:19 I’ve done Python development with and without a formatter and I definitely prefer having the formatter in there. It just eliminates a lot of conversations and it just gets things you say looking very readable. So that was Rustís features and tooling. Let’s talk about frameworks. So, web programming in any language tends to involve frameworks because there’s a lot about web programming that’s similar from project to project and you don’t want to have to implement everything from scratch. So, I’m not sure I totally understood again all of the, roles of the frameworks. But let’s talk about Warp if you think that’s a good starting point in terms of what does it actually do and maybe what the other options were besides Warp today.

Bastian Gruber 00:28:07 So Warp is a web framework. If you come from a different environment, I think that you would, and this is why I wrote in the book as well, I tend to start from the bottom up to see why Rust is different. But if you start from the top, there’s a super small easy to use web framework with it doesn’t come with all the batteries included. I’ve used it in the past. If you want to write a simpler microservice, which you don’t want to have a lot of things included, you want to be small and tiny and quite fast. So, it’s like Warp is built on the Tokio stack and to understand the Rust ecosystem web frameworks, you have to understand that Rust doesn’t come with an HDP server or it even doesn’t come with an abstraction for HDP because it tries to be a systems programming language, it wants to be as small as possible.

Bastian Gruber 00:29:12 It decided to not to implement an abstraction for HDP. So, you need the ecosystem to help you out with writing a web application and for this the, and even to implement asynchronous code functions. So, Warp is built on sort of the Tokio stack. So if you go one level down from a web framework, you need an HTTP server which can run and accepts HDP requests or TCP requests and then it transforms it to HDP messages and Warp is built on top of hyper which is an HDP server implementation in Rust and for it to work asynchronously and then we go one level down, you need a run time. In no JS, you have V8 and GO, you have their own GO run time and in Rust there’s no run time included.

Bastian Gruber 00:30:16 And the run time means some form of threats or threading or how to make sure to not to block if your web server gets one request that it blocks the other 10,000 it might get at the same time. So, you need a way of telling the processor, let’s work on this if you have time and I save the work in some way, and you tell me once you have time again to make work on this and then I pass it on to you. And once you’re done, please notify me and then I can do my part as well. And in Rust the standard run time for this is Tokio which implements this asynchronous thinking mindset on top of the Rust language and there are certain tools built on this Tokio runtime which is hyper-S the HTP server and then the S Warp Esther(?) web framework. So, whenever you use a web framework in Rust you always also have to include which runtime you want to use. It’s not as easy, well it’s I think still pretty simple to have a second dependency to your web framework and this will be most of the time Tokio as the runtime.

Philip Winston 00:31:36 Great. Some other frameworks I think were Actix, Web Rocket and axum. Are those strictly peers of Warp? Are they just one-to-one replacements or are they different altogether?

Bastian Gruber 00:31:51 So with Actix, I think it’s a quite popular framework and this is a very much batteries included type of framework. If you want to start a complex web application and you know you need authentication, you need all the safety measures you know you will need, you might tend to use Actix because there’s a lot of buzz around it, there’s a large community around it, many people are using it. So, it’s a safe pattern not to write everything by hand but you can depend on Actix to already have something for you with Rocket. I think the development pause lately but I’m not so up to date. It’s also very much a batteries included framework but it uses macros quite a lot and macros is a feature in the Rust programming language which you annotate a piece of code and the compiler will transform this piece of code with whatever it’s in the macro.

Bastian Gruber 00:33:02 So, the macro takes your piece of code and will transform it and there are not many rules what a macro can or cannot do. And so, there’s a lot of magic behind the scenes which you either like it or you don’t. And then there’s exome which feels a bit in between Warp and Actix. It’s more, I wouldn’t say easier to use but it’s more from a mindset if you come from like notches (?) you might find it’s easier to think and read about it. So, it’s an in between choice if you’re not too afraid to, yeah to write a middle word yourself or if you’re not too afraid or experienced enough to do a few things yourself then exome is a perfect choice with big enough but not too big to include too much magic.

Philip Winston 00:34:02 So the book uses Warp, is that a reasonable decision for production or was it chosen mostly for the teaching aspect?

Bastian Gruber 00:34:11 So I personally used it in production. The book started with Warp and axum wasn’t out yet. I didn’t want to choose Rocket or Actix because of the teaching part of the book because I think it has too much included and I want to be as close just to like Rust and the concept as possible and don’t want to teach too much about the web framework because it can change and then the book has to change in half a year again. I would maybe choose axum now and axum came out after I was six chapters in, I talked to the maintainer of axum and said, hey how long do you think it will be until this is final? And then it was clear that I’m not going to rewrite half of the book now just to use a different web framework. I’m thinking about updating it or do a web series of how I go through the book and update it to axum. But I would say that I would use axum now for the book. But Warp is perfectly fine for a production environment. You just have to know that you have to do a lot yourself if you want to use it for a complex application. If you just have a smaller microservices, which itself for example fetches a few external resources and power system and gives them back, Warp is perfect for this.

Philip Winston 00:35:35 Okay, that was pretty good overview of frameworks. Now let’s talk about essentially making our first web server. I guess walking through the steps, obviously the book is going to have a lot more detail and a lot more code but just kind of hinting at what we’re doing here. So, I guess the first step might be just setting up a web server with Warp that serves static pages maybe or what would be the very first or just serves a Hello world from the code or something. Is that pretty simple to do?

Bastian Gruber 00:36:06 So the first step as I mentioned is you need to think about the runtime, you need to think about how or which piece of code can handle my asynchronous code. So, you would mark your main function in this example, you put on top of it like in the line above the runtime you’re using with a hash and just a syntax which is used it looks like a macro or is a macro and then you would choose Tokio for it. So, you mark your main function as hey I’m using the Tokio runtime so in a subs subsequent process of the application I can write asynchronous code. So, you would include Tokio and a Warp framework to your dependencies. And then yeah you start or with Warp you start a server and what like Warp is doing is, it has a filter concept.

Bastian Gruber 00:37:17 So each incoming HDP request can get sent through a certain amount of filters and the filters will check, is it a request I can parse? So, with Warp you for example can say this incoming request dot get and then you write the path you expect the get request is for. So, if you have, get request at slash hello but you expect in your path filter slash HelloWorld, the filter would not be triggered. So, then it would say no I’ve expected this at slash hello the HDP request has the path slash HelloWorld. I don’t do much with it I will pass on this request. So, you have to make sure that you add certain filters for your path, for your parameters and then you can add filters like what you want to do with it?

Philip Winston 00:38:22 I think you started talking about route handling. That was going to be my next question. So, the route I guess is like a string from the client with words strings separated by slashes and then optionally at the end you have a query string which is the question mark sort of value equals. So, can you talk a little bit about how the route handler deals with dynamic paths or query strings or anything?

Bastian Gruber 00:38:49 So the one important part about this filter system is always you first have to tell Warp what to look for in this certain filter. So, one route is a combination of the filters and they’re added with an and keyword. So, you have to say if it’s a put like a get or like a post or something and then you say which path is this put for, for example. And then for this path you basically say the slashes and the heart path or the static part of the address and you can say slash questions slash users which is a hard path. And then you have a parameter filter for it as well. So, if you add this with a dot end after the hard path you can say I am expecting a parameter.

Bastian Gruber 00:39:52 So for example if you want to update a certain question and you expect the path as an id, you will say after the question or after the static path you add a dot end and then you say this has to be a parameter and you can give it a type for your application too. So, you can say I want to parse this parameter as a string or as a number for example. And then you close it with an and, and then path end. So, if you now forget to add the parameter in your request, this filter will not be triggered that’s the important part. And then it says no I’ve expected at this static path with a parameter, if I don’t get a parameter this filter is not triggered. And then you can write a recover function for all the requests which were not caught by the filters

Philip Winston 00:40:52 And the query parameters that gets converted to a hash map. Is that true?

Bastian Gruber 00:40:57 So you can dictate for Warp or for your route handler. So, in the path in the parametersí part, it will convert this parameter to a type and pass it as this type to your route handler. So, if you say I’m expecting a hash map or a string or something else, this filter will try to convert this parameter in the path to this type and pass it this type to the route handler.

Philip Winston 00:41:30 So we started off by talking about a web server which generally is going to be returning HTML, you have another chapter in the book I think focused on rest APIs, what are some differences there? And I think part of that section talks a lot about types and creating your own types.

Bastian Gruber 00:41:50 So as with Rust being a type or a strictly type language, you have to before you can work or if you get a request into your web server and you expect in the body some form of Json if you want to work with this you have to tell your web server what’s the type I’m getting from this request? You can just work it as a byte string. You can try to convert this whole thing just into a string, a byte string but you can also say this is a Json and it has to look a struct. Then we talked about strucks in the beginning and there’s a neat little tool called Serde, which can automatically des serialize and serialize for example Json to this construct. So if you want in the book for example add a new question to your database and you add this question as a Json on this HDP request, you can tell in your Rust code now that please transform this Json to this struct and this is done in one line and you can annotate or you have to annotate your obstruct with a macro from this tool called Serde and then your compiler automatically knows and tries to convert this Json, you are getting to this type and if there is a mismatch it says hey this id you told me that’s a number but it’s actually a string I fail.

Bastian Gruber 00:43:32 So this gives you already uh, after the request comes in some hard struck some hard types to work with and with these you can have in the route handler after the filters are done you can say I’m expecting for this route handler in the parameter A type question and the type is your instruct and then you can work automatically with this struck in your Rust code. So, this transformation from struct two and from Json is beautifully done because now we can also compose a new struct and just return it in Warp and survey makes automatically sure that this will be transformed into a proper Jason and this is all done just in one line. So, this makes it really beautiful and really safe to work in in a web environment for an API in Rust.

Philip Winston 00:44:32 Okay, yeah I can see the difference between sort of picking through the Json by field potentially finding or potentially not finding something you need versus upfront converting it to a tight and then you know at that point that it has everything that seems a nice way to go in the rest API section. You did talk a little bit about thread safety. I don’t think we can get into detail but I just wanted to mention this RC and aRc template functions and just what do people need to know about them if they’re going to GO to learn about them later?

Bastian Gruber 00:45:13 So next to the borrow checker, which you need to get used to when writing in Rust. Another concept or another maybe barrier is that you have to dig a little bit deeper because Rust doesn’t have a runtime. You have to think about what it means to that different processes and threats have access to the same data. If you have an in-memory storage, you have to think that multiple threats want to write to the same memory space when you run a web server. So, you have to think about how I pass data and I want to make sure that just one active function or process is able to write to this memory space. And in Rust you usually do two things here. So, you would use and here comes the threat safety and the barrowing at the same time because now it’s a bit complicated to know when should Rust drop a value.

Bastian Gruber 00:46:27 If you have a local storage and you pass it on to a function and then yeah, Rust thinks okay after this is done I can drop it because no other function is going to use it now. But in an asynchronous way you might have hundreds of thousands of, spin up threats or green threats which use the same value. So, you use a concept called an arc, which saves the amount of functions currently holding this value then you can clone it. So if you for example have an in memory storage with all of your questions for this application and you pass it to 10 different route handlers, you would put this in memory storage behind an arc and then you clone this arc which means you duplicate it but for an arc it just increases encounter and says I have 10 active functions currently using this variable so please compile it, don’t drop it.

Bastian Gruber 00:47:42 And after each of the functions are done, it detriments the pointer or the counter and if the counter is at zero, the compiler knows now no one is using this anymore, I can drop it. So now you solve the problem with how to multiply in an asynchronous environment a memory address and pass it to the different route handlers. Now you still need to solve how can I make sure that actually just one function at a time is altering this memory space. And for this you usually have a read write lock or you have a UTEX. So, inside this arc you also pack in a read right lock or a UTEX. So if you have a route handler it gets this arc of this memory storage and then it’s trying to request a lock from the read write lock it says I request a lock because I want to alter this data and then this abstraction will make sure that actually just one function at the same time is altering this data and if two will request a right lock, it’ll make sure that just one of them gets it and the other one has to wait until it’s free again.

Bastian Gruber 00:49:10 So this is why you oftentimes see in this asynchronous environments this combination of arc and read right lock or mute text and then your actual data you pass around.

Philip Winston 00:49:26 Yeah that’s a good example where in the book you have to pause and go pretty deep on some Rust concepts to explain that part of the programming. But I think that’s actually a good way to learn it when you’re motivated for a specific use. So, we have a web server delivering HTML and maybe we have a web service delivering an API results and JSO now another piece is we need to chain and call into someone else’s API. And is there anything to say about that that’s different? I guess in that case you still are using maybe do you make STRs for the API you’re calling into? Is that how it

Bastian Gruber 00:50:08 Works? Yeah, so for the API calls you can also use another crate which in the Rust world is called request it’s a public rate for this and it’s along the same lines where if you fetch a third party service, let’s say a weather service and you want to use this weather data and you expect to chase them back, the beautiful part about Rust is now that some of these APIs can be quite verbose and you might get an object back about the city and the latitude and the weather for each minute and for each hour maybe, but you’re actually just interested in the city name and for this hour, like the weather data, so in Rust and with Serde with the serialize and the de-serialize macro, this one line on top of it, you don’t need to type out the full response, you can just say this struct, let’s call it weather just has a name and a temperature and a date and then it’s perfectly fine to parse this incoming JSON to this one struct and it will just take those fields out, you put them in struct and all the, maybe this incoming JSON has a hundred fields but you just want three of them, you just define the three you want in the struct and then it parses them down just to this one, three and the rest will just be neglected.

Bastian Gruber 00:51:42 So that’s a nice way of not having a too large or two large objects in your code base and two verbose objects. You can just work with a data you want and need.

Philip Winston 00:51:56 All right, one of the other topics in the API section was databases, but we’re going to skip over that here but it was nice to read about starting with the in-memory database and then using I think it was Postgres and there was quite a few examples and information there about that. So, let’s suppose we’ve got our web service implemented. There’s a couple things listed I think under cleaning up your code. One of them was the module system and then logging and tracing and debugging. What can you say about the module system related to web programming or just in general?

Bastian Gruber 00:52:37 I think there’s nothing special about it in Rust. I mean it’s just a way of grouping certain functionalities together and then Rust it’s called a module. So, if you start your application and you see this sort of pieces of code belong together and it shouldn’t be in the main folder, you can create for example a route of a folder where for each endpoint you compose your routes and even maybe have your route handlers there and then maybe you could create a different folder for it called routes. You would add a mod.rs file to it, and this is sort of your starting point into this folder and in this mod.rs file you write out the file names in this folder. So, if this folder has Git questions.rs Git answers.rs, you will write in the mod file PUP Git questions, PUP Git answers. So then other parts of the code can use these pieces of code in this module in main you would then say use and then the folder name and then the name of the module you specified in the mod.rs file. So, it’s just a neat way of separating out your code, which you just, yeah, cleaning up I guess.

Philip Winston 00:54:17 For a web API the difference access points I guess or the different routes nicely, can nicely correspond to modules? How about logging and tracing? I’m trying to remember were those crates, were those third-party utilities or were they built in?

Bastian Gruber 00:54:36 So in Rust- World and on the synchronous Rust-World there is a famous lock crate which sort of works as a facade pattern. So, the lock crate will give you the lock levels as a macro and then you can use these for example info warning in your code. But the actual implementation of this locking functionality is done by another crate. But this second crate is using the lock crate as a facade and providing the actual functionality behind it. So, if you ever want to change your locking crate you can change it but the facade keeps it so you can use the same lines of code and you don’t have to change it, you can just switch out your locking crate without affecting your code base. Now in an asynchronous environment, locking can get messy because it’s not really clear because different tasks on the runtime will finish at different times and will make work on edit at different times.

Bastian Gruber 00:55:52 So your locks might be out of sync or they appear to be at the same timestamp and then it’s hard to say which route handler actually locked this part. And also, in the Tokio ecosystem there is a crate called tracing which allows to annotate asynchronous functions and opens so-called spans, like a span of time so it’s clear which route handler or which threat this log message is for or from. So, if you operate in an asynchronous environment, you always have to think that you can’t control when or how your functions making progress and 10 things can happen at the same time or appear to be at the same time. So, you have to think that your logging might not be as clear in this environment, therefore you have this crate called tracing which adds this few pieces of functionality which can create a span of time and then it makes it a bit more clear where this log is from, from which route handler or from which asynchronous function.

Philip Winston 00:57:08 Let’s talk a little more about bringing things into production and then we’ll start wrapping up. You have a section on authentication and accounts. I don’t want to go through that in detail but is there anything to mention with the authentication is I guess there’s another crate maybe that handles that?

Bastian Gruber 00:57:28 So it depends if you, it depends on the web framework and that’s a good example with Warp you have to think it for yourself. In Actix I think there are a few options you can take off the shelf and implement it and with Warp you have to think about it, how would I do this? There are some crates but I’ve found it depends on the environment but it’s easy enough to actually do it yourself because you’ve expected in a header some form of authentication token whatever you use. And then in the book I just use a different encryption algorithm and therefore I use a library which encrypts this data so I can send it out to the client. But the book shows of how to do it by hand and even if you might not do it in a different environment, I think it’s helpful to know how an authentication actually has to work on a base level and from then on their different crates you can use for different types of environment. I think the book also makes clear that this topic is quite hard to cover in just one book. I think that Manning just has an extra book on authentication Rust in at general and as we all know it’s a very critical thing. So, this was the reason why I wanted to do it by hand to show how to do it by foot so that if you need help or if you want to expand you know how and why and where to look for it.

Philip Winston 00:59:05 Okay. Something else covered was different environments like debug in release, compiling and also running inside Docker images but we’re going to skip over those. I just wanted to touch on testing though before we start wrapping up. Is there some, I think there might have been something built into Rust or Cargo for unit testing or what’s built in there and what do you have to add for testing?

Bastian Gruber 00:59:29 So, Rust comes with a test macro and if you function called assert equal or a sort not equal or a basic set of functionality to write unit tests. So, you can use for example the test macro for function. So, you annotate a function with test and then in your normal production Rust file you can just add a few test functions there. But you can also have a test folder called tests where you write your tests in there and if you run on the command line Cargo test it looks for all these functions which are annotated with tests and runs them. So, this is a nice way of a built-in and it has enough to write your unit tests for. If you go into integration tests and stuff, I think it depends on the flavor of how you want to use them. In the book I explain of how you add a terminology called channels to your web server. So, you can start a server, like the production server in your test environment with certain parameters and then you can use a channel to send a one message to shut it down again if a test fails or if all tests are done and it’s called a one-shot channel and I think that axum has it actually built in which you can use.

Philip Winston 01:01:12 Let’s start wrapping up. What features in Rust or any of the frameworks are you looking forward to seeing in 2023? Is there anything on the horizon that would make web programming easier or anything you’re looking forward to?

Bastian Gruber 01:01:26 I think it made me hopeful and you mentioned this in the beginning the Firecracker. Yeah, written in Rust because so many web services interface with AWS for example that there’s more coming from this direction. I hear from a lot of Node and GO programmers that they mis-crates or they think oh why should I do this by hand? I want a third party to do this for me. So, I hope there will be new releases and a more push into, yeah more Rust flavored third party in the phases. I guess this is a what I look most, most forward to.

Philip Winston 01:02:13 So the book and this conversation we’ve been talking about Rust on the back end. Are you familiar with efforts to run it on the front end and that historically the front end is JavaScript but now with WAM(?) I guess there are ways to run compiled languages. Is this something the Rust community is working towards or have you any familiarity with it?

Bastian Gruber 01:02:36 I think there are two interesting topics around this. Like the one thing which is more in the blockchain world where people run their Rust code in a vasm block in a virtual environment. But also, I was in touch with a company and they do heavy computations and it’s quite expensive to spin this up on AWS and to tear it down again they were thinking of moving this to the edge so to run these types of computations in the browser. So, they looked into shipping these pieces of Rust code in a lop and then the client itself can do the computations in the browser. So, I think this is very, very interesting where you can package up your Rust code and you ship it to a browser. I think that the Docker creator said that if vasm would be out there by the time he wrote Docker, then he wouldn’t have done it because this is actually yeah a great way of doing these types of things. So, I didn’t dig too deep into it yet, but from the use cases I’ve seen is very, very interesting.

Philip Winston 01:03:58 What can you say about the Rust community or the Rust Web community? How would someone best get involved or get started And I’ll put links to anything in the show notes and I’ll also mention, I’ll put links to all of these libraries because some of them are spelled kind of funny. I’ll put them all in the show notes too. But how about community? Are there any central gathering point people should know about or how do they get involved?

Bastian Gruber 01:04:23 So I think they’re multiple if you , my first attempt would always be first look for a meetup near you or for a virtual meetup in your time zone because this is a great way to actually hear people from different companies talking about what they’re doing and then you can chat with real people about your questions. If you, for example, in Europe thereís the Rust Linz meetup, it’s virtual, you can get up early and attend it in the states. And then the, for the more technical questions there is the Rust zulip server, which is more around the actual language progression. They are the working groups gathering there and talking about the next steps in each of these environments. But there’s also the Rust Discord channel, which has different, I guess, rooms to discuss certain things.

Bastian Gruber 01:05:27 There’s a great beginner room you can ask certain questions and the help is always nice and fast. And then each crate or ecosystem, like we talked a lot about the Tokio ecosystem has its own discord channel where you would find hyper Warp axum and Tokio. So if you struggle with understanding certain concepts, you can go there and most of the time the core maintainers of this crate will help answer the questions and generally it’s the nicest environment I’ve worked with sort of, I’m used to JavaScript and JavaScript is quite an open community, but in Rust you have this sort of solid knowledge of people but they always try to help out. So, I would just jump fearlessly into these different Discord servers or onto the meetups and they’re all super nice and open to the newcomers.

Philip Winston 01:06:33 Great. I think that brings us full circle to how you got involved in Rust in the first place. How can listeners learn more about you or the book?

Bastian Gruber 01:06:43 There’s a website called Rustwebdevelopment.com in one word. I try to, or I published two blog posts in the last week. I try to be more active there. I try to publish blog posts there of how to write or rewrite certain chapters in the book from Warp to axum, there are the links I just mentioned on the front page. So, this is the hub or the starting hub to get in touch. I’m on Twitter and that’s the Twitter link I’m on. Yeah, Mastadon there too. And if you want to get in touch via email, you can reach out to foreach@me.com.

Philip Winston 01:07:21 Great. I will put all that in the show notes. Thanks for your time today, Bastian. I enjoyed reading the book and enjoyed talking to you.

Bastian Gruber 01:07:29 Yeah, thank you so much for having me.

Philip Winston 01:07:32 This has been Phillip Winston for Software Engineering Radio. Thanks for listening.

[End of Audio]