We are going to create a web-server in rust that only deals with user registration and authentication. I will be explaining the steps in each file as we go. The complete project code is here repo. Please take all this with a pinch of salt as I’m a still a noob to Rust.
Web and Network Services
All things web and Rust:
- Web applications
- Web sites
- Web browsers
- Web frameworks
- Web assembly
As well as non-web network services.
In this post, I look at how WebAssembly can be used to create serverless functions and demonstrate an AWS Lambda function written entirely in Rust.
This post continues the series on Tide, sketching a possible design for routing and extraction that combines some of the best ideas from frameworks like Rocket, Actix, and Gotham.
The API is written in Rust, a language new to the Pi-hole project. Rust is a safe and fast language which matches well with our goals for the API. It is statically typed and prevents whole categories of errors while being productive and extendable.
Markdown rendering is very important to the performance of Semaphor - every message you send and read is a Markdown document - so we're always looking for ways to improve the performance of rendering Markdown. A couple months ago Jonathan Moore and I wondered how easy it would be to integrate WebAssembly into a React component, replacing the render() function, and we thought that moving Markdown parsing into Rust would be a great way to test this idea out.
WebAssembly is a binary instruction format for a stack-based virtual machine. Wasm is designed as a portable target for compilation of high-level languages like C/C++/Rust, enabling deployment on the web for client and server applications.
Rust has its goals set on to be a primary WASM language and it would be awesome to use it both in backend and frontend web. Ruukh is one of such efforts to realise that dream. Ruukh, a frontend web framework, is inspired by both VueJS and ReactJS.
This blog post is an update to the preceeding article “A web application completely written in Rust” and summarizes the projects’ progress over the last months.
I am proud to announce that I’m working on a book for the Pragmatic Programmers. The title hasn’t been set in stone yet, but the book will be about programming WebAssembly with Rust.
We are pleased to announce the first release of the web-sys crate! It provides raw bindings to all the Web’s APIs: everything from DOM manipulation to WebGL to Web Audio to timers to fetch and more!
Today we are releasing RedBPF and ingraind, our eBPF toolkit that integrates with StatsD and S3, to gather feedback, and see where others in the Rust community might take this framework. If you are looking to up your company’s monitoring game, gather more data about your Raspberry Pi cluster at home, or just have a strong academic interest in Rust and low-level bit shepherding, you might want to read on.
Today, Tower Web 0.3 has been released and it comes with two major new features: Experimental support for async/await. Support for using templates to render responses.
Smithy, a web development framework written in Rust that compiles to WebAssembly. In the mean time, there has been substantial progress, and Smithy is on the verge of being ready for alpha use! In this post, I want to describe the improvements that have been made, and what’s on the Smithy roadmap!
The 2.0 release of Linkerd brings two very significant changes. First, we’ve completely rewritten Linkerd to be orders of magnitude faster and smaller than Linkerd 1.x. Linkerd 2.0’s data plane is comprised of ultralight Rust proxies which consume around 10mb of RSS and have a p99 latency of <1ms. Linkerd’s minimalist control plane (written in Go) is similarly designed for speed and low resource footprint.
The Network Services Working Group aims to improve the story for web development this year in several respects: by bolstering foundations like async/await, by improving the ecosystem of web-related crates, and by pulling these pieces together into a framework and book called Tide. The name “Tide” refers to “a rising tide lifts all boats”; the intent is to improve sharing, compatibility, and improvements across all web development and frameworks in Rust.
Let’s take a look at how in Rust you can have an algorithm generic over T, where T is further bound by a trait, which itself is generic over one of several parameters.
tower-web version 0.2.2 has been released. It comes with a number of new features, which I will talk about in this post. Primarily, the middleware story is starting to come together. I will be expanding some on how middleware fits into Tower and web in general.
I set out out my goal 9 for Rustnish: Write benchmark code that compares runtime performance of Rustnish against Varnish. Use cargo bench to execute the benchmarks.
The basic idea of a performance test here is to send many HTTP requests to the web service (the reverse proxy in this case) and measure how fast the responses arrive back. Comparing the results from Rustnish and Varnish should give us an idea if our performance expectations are holding up.
In Servo, task-sources are implemented via a channel, whose sender is cloned for each specific task-source, and where tasks are messages sent on the channel and containing a closure representing the actual task.
In "Oxidizing sourmash: Python and FFI" I described my road to learn Rust, but something that I omitted was that around the same time the WebAssembly support in Rust started to look better and better and was a huge influence in my decision to learn Rust. Reimplementing the sourmash C++ extension in Rust and use the same codebase in the browser sounded very attractive, and now that it was working I started looking into how to use the WebAssembly target in Rust.
Today, let’s go through an entire fetch in Servo, starting with an example in JS.
Rust's web frameworks ecosystem is in constant change, but recently a new framework called warp came out implementing a new, original way to solve the old problem of transforming a request into a response, and I wanted to give it a try.
And, as I use GraphQL massively at work, I also wanted to check how well Juniper implements it. To add some spice, I used MongoDB as a storage engine instead of the ubiquitous and well-supported SQL databases.
A sensible, modern pastebin written in Rust
The short version is, Tower Web 0.2 was just released and regular Rust attributes are now used instead of magic comments. The doc comment is replaced with #[get("/")]. This is thanks to Rust macro wizard David Tolnay. I also thought that it would be best to immediately push out 0.2 and then we can all pretend 0.1 didn’t happen.
This is a demonstration of a pure-rust library to generate syntax diagrams for macro_rules!().Diagrams are generated fully automatically from rust-source as Scalable Vector Graphics, using customizable CSS for layout.
I previously announced Tower and mentioned that a web framework was in the works. It took longer than I had hoped (as it sometimes does with software), but today, I am opening up Tower Web.
Tower Web is an asynchronous HTTP web framework that focuses on removing boilerplate. It is built on top of Tokio, Hyper, and of course Tower. It works today on stable Rust.
In the first article of this series, we created a RestAPI for our application using Postgres, Rust, and Rocket. For the second half, we’ll be using React to create the front end.
Over the past several months, I’ve been working a web framework in Rust. I wanted to make use of the new hyper 0.12 changes, so the framework is just as fast, is asynchronous, and benefits from all the improvements found powering Linkerd. More importantly, I wanted there to be a reason for making a new framework; it couldn’t just be yet another framework with the only difference being I’ve written it. Instead, the way this framework is used is quite different than many that exist. In doing so, it expresses a strong opinion, which might not match your previous experiences, but I believe it manages to do something really special.
I’m super excited to reveal warp, a joint project with @carllerche.
For this post, I’m going to make three comparisons: to Flash, to Java Applets, and occasionally to PNaCL. Secondly, this post is going to focus on the web use-case for WebAssembly, even though the previous post was about non-web uses. We’ll make that comparison in the next post. Finally, this post is kind of like eating tapas, there’s a bunch of little sections.
A three day tour of Yew and WASM with Rust
Over the past few months, I’ve been writing Smithy, a very work-in-progress front-end WebAssembly framework written in Rust.
My goal for Smithy is to enable you to use idiomatic Rust to write front-end code. This has costs: for example worrying about lifetimes and using Rc
> to share state. But this also has the potential to give you the safety guarantees that the Rust compiler provides when writing browser code!
The idea was simple, choose a package and recursively traverse npm to find all of the packages that depend upon it, after all the npm website offers a ‘Dependents’ tab on a package’s page. I had not anticipated the issues that made this an interesting technical problem.
We've just finished music festival Atlas Weekend which took place in Kyiv, Ukraine. This year there were for about 450,000 visitors during 6 days. I'm glad to announce that Rust and actix-web are used as a backend for main technical purposes of festival.
In our last post about actix we introduced you to the Rust programming language and the actix actor framework. This week we will build a basic TCP client with actix.
My latest software architectural experiment is to write a complete real-world web application in Rust with as less as boilerplate as possible. Within this post I want to share my findings with you to answer the question on how much web Rust actually is.
The last blog post laid out the Rust and WebAssembly domain working group’s overall vision for Rust and WebAssembly. In this blog post, we will dive into the details of wasm-bindgen, the future we envision for it, and how you can help us build that future.
A tutorial on how to compile Rust to WebAssembly and integrate it into a React application
This is the first part in a two-part introductory series to PRRR Stack (Postgres, Rust, Rocket, React) application.
In our previous post, we saw how combining channels with an event-loop could be a useful technique to ‘drive’ the concurrent logic of your system, and it was hinted at that shared mutable state might be more complicated. A good example of such ‘shared mutable state’ in Servo is the HTTP cache.
I had this goal of making a web app that was very fast, stable and easy to deploy. This is what I was hoping to accomplish: strongly typed server-side and client-side languages (Rust & Elm respectively), push-button deployments to the cloud with free hosting initially, and sub-second response times for API calls and page loads. With this setup, I think I'm well on my way to accomplishing each of these.
Better HTTP Upgrades with hyper It’s been possible to handle HTTP Upgrades (like Websockets) in hyper if you made use of the low-level APIs in the server and client, but it wasn’t especially nice to...
This game was coded in Rust and is playable in web browsers by means of WebAssembly, WebGL and Howler.js. The software I developed is partly open source in the form of Gate, which is the Rust library that powers this game and can power other similar games. Special thanks to the tools I used to create assets: Gimp, FL Studio and BFXR.
Rust and WebAssembly can combine in many delightful ways. To consolidate our efforts and have the largest, most-positive impact we can, the Rust and WebAssembly domain working group is focusing on one vision:
This blog post will expand on these aspirations and describe where we stand in relation to them right now. In a series of follow up posts, we will talk about the next steps for each major component of the Rust and WebAssembly ecosystem.
Lately I’ve been working on a side project that involves aggregating the indexes of media libraries and allowing one library to “lend” a file to another. For example, I might use this system if I want my home tower to “lend” a file to my laptop before I get on a plane. I’ll be committing my work-in-progress on this project to github shortly (just want to get an MVP working first and do some refactoring), but I thought I would take a step back for a moment and share my thoughts on one aspect of this project: gRPC. In particular: gRPC streaming.
The 3rd of a multi-part series where we build a small REST client for the Marvel Comics Web API using Rust. This time we focus on how to use the hyper and futures crates to make HTTP requests to a Web API, then use serde to parse JSON data from the response.
Welcome to the third post in a series where I share my experience learning Rust. I’m building out a conspiracy theories API to help me get more familiar with Rust and have a little fun. Since I am new to Rust, I welcome all feedback, especially from developers who have been using Rust for quite some time. Leave a comment below or send your feedback via Twitter. Now it’s time to draw the blackout curtains, put on your foil-lined hat on and start serving up conspiracies from a REST API.
The “actor model” is the main primitive that powers the Erlang programming language and its descendant, Elixir. It describes a programming model that simplifies the development of concurrent and multi-threaded applications or even applications that run distributed on multiple machines.
The complexity of actors is relatively low, and that is because the complexity is usually hidden in the actor frameworks that are used to run these types of primitives in the end. One example of such an actor framework is actix, which we will have a closer look at now.
I was inspired by Nick’s post to migrate my code from Rocket to Actix. I have also been nagged by recent struggles with the nightly compiler and its compatibility with all the other crates that I use and other the platforms that I use; my code runs on 32⁄64 bit arm and 32⁄64 bit x86. When I started using Rocket around a year ago, the nightly compiler was just as stable as the stable compiler, but things have changed as rust is a growing eco system.
For me, once the differences below were addressed, porting the code was relatively straightforward. The vast majority of my code remained the same.
Creating a simple web service for automating Travis builds across repositories using actix-web and Github webhooks.
Today sees the release of hyper v0.12.0, a fast and correct HTTP library for the Rust language.
This release adds support for several new features, while taking the opportunity to fix some annoyances, and improve the extreme speeds!
While the http crate generally has a great API I have been unsatisfied how it handles URLs. To create a HTTP request a full URL is needed with a scheme (http/https), authority (example.org) and a path (/search?q=rust) but http does enforce this and allows you to only state the path. This means both clients and servers are either unable to determine protocol and and authority information or have to do this manually.
A little while ago I came across this Snowflake generator. It’s a project by Raph Levein that takes a hash string and uses it to procedurally generate a unique snowflake. He explains that the original motivation was as a cryptographically secure visual hash, so that people would reliably be able to tell by visual inspection whether two hashes were identical. I thought that was a pretty cool idea.
I write code for the web every day, and as such, I naturally wanted to find a web server that I could write some beautiful code in Rust with. There are a few options out there, Actix, Rocket, but none of them are quite as simple and succinct as KoaJS for node. As such, I wanted to write something as simple as Koa and as performant as a Rust framework, and I did. With that, I give you Thruster, an elegant, performant, web server written in Rust.
The 2nd of a multi-part series where we build a small REST client for the Marvel Comics Web API using Rust. This time we focus on how to use the hyper and futures crates to make HTTP requests to a Web API, then use serde to parse JSON data from the response.
I previously wrote an article back in November 2017: Replacing Elasticsearch with Rust and SQLite. In it, I needed to create a few HTTP endpoints that ingested JSON, perform a database lookup, and return JSON. Very simple. No query / path parameters, authentication, authorization, H2, or TLS. I didn’t understand how stability was such an important feature. I was familiar with needing new versions of the nightly compiler to stay current with clippy and rustfmt, but it was a blindspot when it came to dependencies.
Here I go with my first post that fully focuses on Rust. After spending a few months doing a bit here and there I decided to just dive right in as I was going through the Rust book at too slow a pace to keep myself interested. So, in this post I decided to write about setting up a simple REST API which is something that I have done in Java plenty of times but with Rust it is a different story.
I had the need for a very small API for this website. So small, in fact, that only one endpoint was required. I've been doing a lot of development in Rust lately, so naturally Rust seemed like a good candidate to build this API in. I also wanted to try out a newer Rust web framework called Actix web. It claims to be a "small, pragmatic, and extremely fast rust web framework", which sounded perfect for what I needed.
This is a small idea that I have been wanting to put together for quite some time now and finally have managed to get the time and most importantly experience in Rust to finally try something a little more than just small projects. One area that I think Rust is really making a decent headway in in the web domain, which I am assuming is likely due to its origin from Firefox. So I wanted to see if I could put together a really basic CRUD micro-service doing the ever so original TODO functionality.
Using Rust on Microsoft Azure Functions with web assembly
A project used for a meetup talk about getting started with rust & wasm
Hopefully this post helps jumpstart those who want to use LLVM from Rust. I also include how one could use this to generate WebAssembly, but the first part of this post is target platform agnostic.
C, C++, and Rust all have some capability for dynamic dispatch: function pointers, virtual methods, and trait objects. On native targets like x86, all these forms compile down into a jump to a dynamic address. What do these forms compile down into when targeting WebAssembly?
wasm-pack is a tool for assembling and packaging Rust crates that target WebAssembly. These packages can be published to the npm Registry and used alongside other packages. This means you can use them side-by-side with JS and other packages, and in many kind of applications, be it a Node.js server side app, a client-side application bundled by Webpack, or any other sort of application that uses npm dependencies. You can find wasm-pack on crates.io and GitHub.
I wrote the NES emulator with Rust and WebAssembly to learn Rust. It’s not perfect and have some audio bugs, but it’s good enough to play Super Mario bros.
Rust aims to be the foundation of fast and robust software. A thorn in that side is the choice of timeout-centric APIs for I/O and multithreading. I posit this was a bad choice and it needs to be remedied as soon as possible before more APIs proliferate this (possibly accidental) design decision.
Yew is a new web framework written in Rust that takes concepts from React JS and Elm. It takes the two web libraries I love and combines them with the new language I love — Rust. I wrote this article because I haven’t found a lot of documentation online on how to get the Yew examples running, and I’m sure there will be people that will run into the same problems I did. I wrote this in hopes that it helps someone out there.
Serving content from a Rust web server running on a Raspberry Pi from your home to the world, with a Cloudflare Argo Tunnels.
Hello and welcome to the first issue of This Week in Rust and WebAssembly! Rust is a systems language pursuing the trifecta: safety, concurrency, and speed. WebAssembly is designed as a portable target for compilation of high-level languages like C, C++, and Rust, enabling deployment on the web for client and server applications. This is a weekly summary of its progress and community.
Tower is a library for writing robust network services with Rust. It is being built in service of the Conduit proxy, which is using the Tokio ecosystem to build the world’s smallest, fastest, most secure network proxy. Tower will also provide a batteries included experience for implementing HTTP and gRPC services.
Please read the Part 1 of the series here :
For one of our customers at Centricular we were working on a quite interesting project. Their use-case was basically to receive an as-high-as-possible number of audio RTP streams over UDP, transcode them, and then send them out via UDP again. Due to how GStreamer usually works, they were running into some performance issues.
This blog post will describe the first set of improvements that were implemented for this use-case, together with a minimal benchmark and the results. My colleague Mathieu will follow up with one or two other blog posts with the other improvements and a more full-featured benchmark.
The short version is that CPU usage decreased by about 65-75%, i.e. allowing 3-4x more streams with the same CPU usage. Also parallelization works better and usage of different CPU cores is more controllable, allowing for better scalability. And a fixed, but configurable number of threads is used, which is independent of the number of streams.
The newest release of hyper includes some lower-level connection APIs for both the server and client. Notably, this allows using hyper send and receive HTTP upgrade requests. The most popular of these is Websockets.
I started writing mob, an multi-echo server using mio, in 2015. I coded mob into a mostly working state and then left it mostly alone, only updating it to work with the latest stable mio. Recently, I started looking at the code again and had the urge to improve it. In a previous post, I talked about managing the state of connections in mob. In this post, I will walk through what I did to remove the need to track connection state. I wanted to remove the state because the implementation required an O(n) operation every tick of the mio event loop. It also added a fair amount of complexity to the code.
Blazing fast, low requirements, computationally intensive operations on Node.js using Rust
Central to the problem are the edges. People will reliably do a good job of building and testing the happy paths, but as humans we’re terrible at considering the edge conditions, and it’s those edges and corners that cause trouble over the years that a program is in service.
Since my initial Node/Rust REST comparison, I’ve wanted to follow up with a comprehensive guide for getting simple CRUD operations up and running in Rust.
A brief post to help others multicast in Rust
With the combined goal of gaining a deep understanding of DNS, of doing something interesting with Rust, and of scratching some of my own itches, I originally set out to implement my own DNS server. This document is not a truthful chronicle of that journey, but rather an idealized version of it, without all the detours I ended up taking. We'll gradually implement a full DNS server, starting from first principles.
WebAssembly has been interesting me lately, specifically the prospect of doing arithmetically-intensive operations with it, not unlike the fantastic physics engine Emscripten port Ammo.js. Compiling something like that is out of the scope of this little post (but should be getting easier!), and I’ve left some links at the bottom so you can clear more about WebAssembly if you’re interested.
I’m writing a reverse proxy in Rust using Hyper and I want to measure performance a bit to know if I’m doing something terribly wrong. By doing that I discovered a Denial of Service vulnerability in Hyper when IO errors are not properly handled. Note that a workaround has been released in the meantime in Hyper 0.11.20, more background info can be found in this Hyper issue.
IPFS is a peer-to-peer protocol that allows you to access and publish content in a decentralized fashion. It uses hashes to refer to files. Short of someone posting hashes on a website, discoverability of content is pretty low. In this article, we’re going to write a very simple crawler for IPFS.
It’s challenging to have a traditional search engine in IPFS because content rarely links to each other. But there is another way than just blindly following links like a traditional crawler.
I’ve been writing a small toy project with Tokio in my spare time’s spare time. I’ll write more about it at a later date. What I’ve found writing it, though, is that there is a specific pattern to writing servers, both UDP and TCP (and presumably others) in Tokio, and it’s not super obvious at first glance. So here it is.
Let me begin this article on Writing a Microservice in Rust by talking about C++. I’ve been a reasonably active member of the C++ community for quite a while now, attending and contributing talks to conferences, following the development and evangelism of the language’s more modern features and of course writing lots of it. C++ is a language that gives its users very fine-grained control over all aspects of the program they are writing, at the cost of a steep learning curve and a large body of knowledge required to write effective C++. However, C++ is also a very old language, conceived by Bjarne Stroustrup in 1985, and thus caries a lot of technical debt even into modern standards.
Vyacheslav Egorov, who goes by mraleph on the Web, wrote a response to my article “Oxidizing Source Maps with Rust and WebAssembly” titled “Maybe you don’t need Rust and WASM to speed up your JS”.
It’s been a few months since I shifted my focus full time to Rust, and in that time, we’ve gotten a lot of work done! I wanted to update you here what all that is, since it’s spread around on multiple repositories.
I recently spent some effort trying to make reproto run in a browser. Here I want to outline the problems I encountered and how I worked around them. I will also provide a number of suggestions for how things might be improved for future porters.
Fanta is a web framework that aims for developers to be productive and consistent across projects and teams. Its goals are to be: Opinionated, Fast, and Intuitive. Based heavily off of the work here: https://github.com/tokio-rs/tokio-minihttp
My nice brother Johannes Ridderstedt sent me some old files a few weeks ago (in late 2017), stuff that he had preserved from an age-old computer of ours. One of these was the file named gameland.zip (not published yet, but I might put it up here some day.) I managed to get this running, and liked what I saw (you'll find the YouTube link to it further down on this page.) Around this time I was reading a bit about WebAssembly which I think will redefine and help reshape the web as we see it today. I was also looking at the Hello, Rust web page, and the "FizzleFade effect using a Feistel network" page in particular.
Everyone loves event sourcing, right up until the moment they have to implement it. At that moment all the wonderful whiteboard drawings…
I woke up this morning with one goal: to convert my website (morgangallant.com) into a Rust web server. I have been a long time fan of…
Introducing wee_alloc. wee_alloc is a work-in-progress memory allocator designed for WebAssembly. It has a tiny code size footprint, compiling down to only a kilobyte of .wasm code.
概要 以前、JSで書いた(ファミコンのエミュレータを書いた - undefined)ファミコンのエミュレータをRustで書き直してみた。 また、技術的な内容はQiitaの方にも書いているので興味のある方は参照してみてください。(まだ Hello, World!までしか書けてませんが。) qiita.com もともとファミコンのエミュレータって新しい言語を習得するのにちょうどいい題材だったりするのでは、って話しからスタートしてて、よくわからないのでJSで書いてみて、ようやくRustで一通りは実装できた感じ。まだバグや未実装（音声周りやマッパー）も多いんですが、ひとまずはお腹いっぱいな感じ。 成果…
DataFusion is an open-source Big Data platform implemented in the Rust programming language with a similar programming style to Apache Spark.
In this post, I'll walk through a few of the highlights of getting Turtle, a Rust library for creating animated drawings, to run in the browser with WebAssembly.