The PingCAP company, makers of the TiDB NewSQL database and the TiKV key-value store, have kindly made publicly available, as well as open-sourced, a set of training courses that they call the "PingCAP Talent Plan". These courses train programmers in writing distributed systems in the Go and Rust languages. They are originally intended by PingCAP to train students, new employees and new contributors to TiDB and TiKV and focus as such on subjects relevant to those projects, but are still appropriate to anyone with an interest in learning to make distributed systems in Go and/or Rust.
Web and Network Services
All things web and Rust:
- Web applications
- Web sites
- Web browsers
- Web frameworks
- Web assembly
As well as non-web network services.
In the previous post How to modulize your Rust Frontend, we learnt how to use impl, functions and Yew components. They help you to find errors and organize your Rust frontend project. In this post, we will include server side code with ws-rs. It will help us to build complete Rust Full Stack chat app similar to what we made at How to start Rust Chat App.
In the previous post How to use markdown with Rust Frontend, we learnt how to render markdown in Rust frontend and include CSS files from your previous frontend project. You can use a text, image, video, markdown or any HTML elements you want for your Rust frontend. In this post, we will learn how to modulize your Rust Yew frontend app. It will be easy after you learn how to use impl, function, and components for that.
After taking a break from working on Arrow and DataFusion for a couple of months to focus on some deliverables at work, I have now started a new PoC project to have a second attempt at building a distributed platform with Rust, and this time around I have the advantage of already having some foundational pieces in place, namely Arrow and DataFusion. I have also been gaining experience with Kubernetes recently and I could clearly see how this would simplify the creation of a distributed platform. The pieces really are starting to fall into place.
The new project is called Ballista and is a fast moving PoC taking a top down approach to building a distributed platform.
In this article, we’ll implement and deploy a Gotham full-stack web framework using the Tera template system, Webpack for a complete front-end asset management, a minimal VueJS and CoffeeScript web app and deploy to Heroku. Gotham is a Rust framework which is focused on safety, speed, concurrency and async everything. Webpack is a NodeJS website asset preprocessor and bundler which can let you use any of your favorite front end technologies. Combining these technologies allow for a small footprint on the server, which means saving money on infrastructure, very fast performance in page load for higher visitor retention and the full flexibility of client side code available to you for web design without limitations.
In this blog series, I will experiment with Rust as a safer and simpler C/C++ replacement. The idea is to combine a couple of C dependencies in Rust, to do some work using the dependencies in Rust and to expose a final API from a Rust library via C ABI. Then I will consume the same exported Rust methods from a number of modern languages/platforms such as C#, Python, Java, Nodejs, R, Go and even back from C/C++.
I'm happy to announce, that we've just finished one of the biggest European music festival Atlas Weekend which took place in Kyiv, Ukraine. As a year ago, backend for technical purposes of festival is written with Rust: Tickets exchange, entrance control, powerful user management with distributed configurable permissions, backstage pass, control of security workers, their work time, cars entrance, warehouse system for rfid bracelets with full history and control of each rfid. This year we registered more than 6000 workers, each of them has full info tied to bracelet: name, phone, photo, available permissions, time to access festival.
In the previous post full-stack Rust with Yew, we learnt how to prepare minimal files to build a full stack Rust web app. You can do whatever Rust allows with it.
In the previous post How to use Rust Yew, we learnt how to prepare minimal files to build webassembly files with Yew for Rust frontend. We will advance it with some Rust server side code and write a bash file to automate the process.
Creating a simple console application in Rust for fun and profit, including some nice tricks to push async programming to the limits.
Dev's offline page is fun. Can we do that with Rust and WebAssembly? The answer is yes. Let us do it.
In this post, we will prepare development environment for Rust Yew. Then, we will write minimal code with it and learn how to deploy it in your website also.
What are we gonna do? Create a WebAssembly application that takes a string in markdown format and converts that into HTML.
In this article we are going to create a WebAssembly Hello World program with Rust. You might be thinking that a WebAssembly Hello World tutorial could be found almost everywhere, including one on Mozilla website and rustwasm.github.io.
It’s been a year since I wrote a little proxy with Rust, it is one of my first projects with this language and I learned a lot writing it. To sum up a bit, the main goal of this proxy is to be really simple to use and easy to extend with middlewares. It targets HTTP APIs and can be used in front of a lot of services. This proxy has been used in multiple projects, with microservices architecture and so on…
A hands-on introduction into WebAssembly ( Wasm ). Containing simple examples and tutorials on how to implement concepts and various tasks using Wasm.
As with most hobby projects, my htsp-rs implementation became a side-side-side project, and it will probably stay that way. Another property of those hobby projects is that I now-and-then pick them up again, and htsp-rs has come to that point now. The reason for picking it up again is different from the reason of its existence though, and that’s a good thing: it means I have multiple reasons for the crate’s existence. Initially, I wanted it as a back-end for a livestreaming app on SailfishOS. Today, I want it as back-end for mpd’s TVHeadend support.
We are going to create a web-server in rust that only deals with user registration and authentication. I will be explaining the steps in each file as we go. The complete project code is here repo. Please take all this with a pinch of salt as I’m a still a noob to rust 😉.
In a previous post, we created an Actix 0.7 Web App, which was not fully non-blocking. In this post, we'll become fully non-blocking and upgrade the app to Actix 1.0
I've been experimenting with techniques for collaborative editing in Rust recently and I'd like to share my first functional prototype with you. The demo uses Rust and WebAssembly, and I implemented it using a conflict-free replicated data type (CRDT) based on LOGOOT, which I modified to work with variable-sized strings rather than individual characters or lines.
The vision is to build a livecoding / design hybrid program, where procedural design and code are fused in one environment. If you have missed 'learnable programming' please check this out: http://worrydream.com/LearnableProgramming/ Makepad aims to fulfill (some) of these ideas using a completely from-scratch renderstack built on the GPU and Rust. It will be like an IDE meets a vector designtool, and had offspring. Direct manipulation of the vectors modifies the code, the code modifies the vectors.
However before we can make this awesome application, we need to build a UI stack. The aim of this toolkit is to be our stepping stone into building a livecoding IDE and designtools that don't suck or fall to pieces along the way.
Lightbeam is a new streaming compiler for WebAssembly, designed to produce the best possible assembly while still being fast enough to produce assembly faster than the WebAssembly is received over the wire.
Actix web 1.0.0 is released - a small, pragmatic, and extremely fast web framework.
I find Gopher really cool. I think it’s a really nice way to organize information into trees and hierarchies, and as we all know programmers can’t resist trees. So recently I took an interest in Gopher and started writing my own server.
But recently it’s been gaining traction; so we should provide a decent landscape for new gophers, full of oxidised servers. Since I started using Gopher more often, it’s beneficial for me if there’s more content out there. So I’m writing this blog post to walk you through how to write your own server. We’ll be doing this in Rust.
In this series, we are creating a basic static HTTP 1.0 server with Rust. If you haven’t seen Part 1 yet, go do that first. At the end of Part 2, our server will do the following: Read and serve files from a predefined directory on the host server, Generate appropriate HTTP responses to incoming requests, Log information about the response to standard output.
It all started with a simple issue, a “quick fix”, or so I thought.
General purpose computation on the blockchain using Web Assembly System Interface (WASI).
This is the story of our journey running Wasmer on the Linux kernel.
In this series, we will create a basic static HTTP 1.0 server with Rust. At the end of Part 1 of this tutorial, our server will do the following: Listen for and handle TCP connections on a specific port, Accept HTTP 1.0 GET requests, Parse and validate requests for further use, Log incoming requests. We will avoid using libraries that make this trivial (i.e. the http crate) and focus on the fundamentals of how a server works.
Stretch is a cross-platform Flexbox engine written in Rust. At Visly we are building a design tool for front-end engineers and we needed to ensure components looked the same across web, iOS, and Android without making use of WebViews. This meant replicating the web layout system on mobile.
In this post, I’ll cover the test setup we use in Stretch, how and why we need to generate unit dynamically. I’ll also cover an example of contributing another test to Stretch, and finally I’ll walk through how we also make use of this system for benchmarking.
When writing a web service, I often lean towards using tools that are as minimal as possible. One pretty obvious reason for this is the avoidance of dependencies you either don't want or don't need in your project. Whilst I'm not someone who goes out of their way to avoid dependencies, this is a pain point particularly in Rust because of the cost of building them repeatedly rather than shipping extra interpreted files around (especially if you use "pure" build environments).
This article is more of a how-to on getting up and running with a production-grade web project which incorporates Rust (or any other language for that matter) and WebAssembly into your web pages.
If you are coming from NodeJS, Futures in Rust don't make much sense. In NodeJS, everything happens asynchronously. Therefore for you to be able to say "Hey, I really need to wait for the answer for this GET HTTP call", you are putting.then() on a Promise, so you can make sure you just execute the code inside the .then() when the HTTP call is finished.
In Rust, everything is blocking and synchronous by default, so you might ask yourself: "Why bothering with the complexity, that's exactly what I wanted in the first place!"
Build a WebAssembly application with a hash router using Smithy.
In the last three posts of this series we covered all of the things we would need to use Wasmer as the base for a plugin system. In part one we went over the basics of passing simple data in and out of a web assembly module, in part two we dug deeper into how you might do the same with more complicated data. In the last part we eased the experience of plugin developers by encapsulating all of our work into a library that exports a procedural macro. In this post we are going to explore what it would take to extend an existing plugin system to allow for wasm plugins.
Rust is one of the newest kid on the block in the modern programming languages. We tried to take advantage of its great features by using it in an Image Recognition blueprint project. This project can be traded with any existing security system deployed at any organization. Hawk uses AWS services integrated with Rust.
Using sonr-extras to build a very basic chat using the provided connection handling objects.
ZEIT is proud to announce official support for Rust on Now through `@now/rust`. Our mission at ZEIT is to make cloud computing accessible for all. Rust has seen rapid growth in its adoption, and we're proud to be able to support the community.
In the last two posts of this series we covered all of the things we would need to use Wasmer as the base for a plugin system. In part one we went over the basics of passing simple data in and out of a web assembly module, in part two we dug deeper into how you might do the same with more complicated data. In this part we are going to explore how we might ease the experience for people developing plugins for our application.
In this post we are going to cover how we could pass more complicated data from the wasm module back to the runner.
Plume is a federated blogging engine, based on ActivityPub. It uses the Rocket framework, and Diesel to interact with the database.
A few months ago, the Wasmer team announced a Web Assembly (aka wasm) interpreter that could be embedded into rust programs. This is particularly exciting for anyone looking to add plugins to their project and since Rust provides a way to directly compile programs to wasm, it seems like a perfect option. In this series of blog posts we are going to investigate what building a plugin system using wasmer and rust would take.
The most common pattern for creating APIs is REST. We will discover how we can build an API in Rust which conforms with the REST pattern.
Last article I wrote about how to use tensorflow with rust. This time we're going to take what we've built on, and serve it as an HTTP API call. As Actix Web is nearing its inevitable 1.0 release, I thought it would be a great time to build something with it.
Today, we're open sourcing and announcing wrangler, a CLI tool for building, previewing, and publishing Rust and WebAssembly Cloudflare Workers. If that sounds like some word salad to you, that's a reasonable reaction.
As a first little project in Rust I thought I'd do something familiar, so I created a small web application, which we'll check out in this post.
In this tutorial, we'll create web app using the Rust actix-web framework and implement magic link authentication powered by ApproveAPI's Rust library.
Linkerd 2.0 introduced a substantial rewrite of the widely adopted service mesh, using a split between Go and Rust. In this article, we discuss the lessons learned in the "cauldron of production adoption", and how those lessons became the basis of Linkerd 2.x’s philosophy, design, and implementation.
I’m extremely excited to announce the 0.0.2 release of Smithy, a web development framework for Rust! While it is a very pre-alpha version, it should be functional enough for others to start playing around with. Please, get your feet wet and provide feedback.
Today, we are thrilled to announce the open sourcing of Lucet, Fastly’s native WebAssembly compiler and runtime. WebAssembly is a technology created to enable web browsers to safely execute programs at near-native speeds. It has been shipping in the four major browsers since early 2017.
Lucet is Fastly’s native WebAssembly compiler and runtime. Using the Lucet runtime and Rust’s wasm32-unknown-wasi target, we can create a WASM program that runs on the server.
Ocypod is a language-agnostic, Redis-backed job queue server with an easy to use HTTP interface. Its focus is on handling and monitoring long running jobs.
tarssh is an SSH tarpit — a server that trickles an endlessly repeating introductory banner to clients for as long as it remains connected, in order to expend the resources of attackers. It's based on the same concept as Chris Wellons' Endlessh, a similar service written in C.
I'm distinguishing Seed through clear examples and documentation, and using wasm-bindgen/web-sys internally. I started this project after being unable to get existing frameworks working due to lack of documented examples, and inconsistency between documentation and published versions. My intent is for anyone who's proficient in a frontend framework to get a standalone app working in the browser within a few minutes, using just the quickstart guide.
Seed's different approach to view syntax also distinguishes it: rather than use an HTML-like markup similar to JSX, it uses Rust builtin types, thinly-wrapped by macros that allow flexible composition. This decision will not appeal to everyone, but I think it integrates more naturally with the language.
Using Rust and AWS Lambda to thumbnail images, cheaply, relablity, and quickly.
If you learn something new, always have an updated version of your project in production. It keeps you motivated. We cover the most common options of how to do this in and with Rust.
Dodrio is a virtual DOM library written in Rust and WebAssembly. It takes advantage of both Wasm’s linear memory and Rust’s low-level control by designing virtual DOM rendering around bump allocation. Preliminary benchmark results suggest it has best-in-class performance.
WASP is ‘a LISP programming language for extremely performant and concise web assembly modules.’ That means that you can use WASP to generate WASM modules.
Almost by accident, it turned out that I implemented a pretty simple, but non trivial task in both C and Rust and blogged about them. Now that I’m done with both of them, I thought it would be interesting to talk about the differences in the experiences. The Rust version clocks at exactly 400 lines of code and uses 12 external crates. The C version has 911 lines of C code and another 140 lines in headers and depends on libuv and openssl.
Wasmer is a WebAssembly runtime designed to run both standalone and embedded. The crate wasmer-runtime exposes an easy to use and safe api for compiling, creating imports, and calling WebAssembly from your own library. This tutorial goes over how to make a simple wasm application and run it using the wasmer-runtime!
During last year’s Birthday Week we announced early support for QUIC, the next generation encrypted-by-default network transport protocol designed to secure and accelerate web traffic on the Internet.
We are not quite ready to make this feature available to every Cloudflare customer yet, but while you wait we thought you might enjoy a slice of quiche, our own open-source implementation of the QUIC protocol written in Rust.
At HAL24K, we benefit a lot from open source software. That is why, to contribute back, we’ve started an internal program to open source some of the internal tools and libraries we’ve used to build our platform and machine learning solutions, starting with OOProxy. OOProxy is a reverse OpenID and OAuth2 proxy that we use to protect our HTTP-based machine learning APIs.
In my previous articles I talked about how WebAssembly allows you to bring the library ecosystem of C/C++ to the web. One app that makes extensive use of C/C++ libraries is squoosh, our web app that allows you compress images with a variety of codecs that have been compiled from C++ to WebAssembly.
In my experience, most performance problems on the web are caused by forced layout and excessive paint but every now and then an app needs to do a computationally expensive task that takes a lot of time. WebAssembly can help here.
In this article about the state of the Rust and Wasm ecosystem, I'll try to explain why Rust is the language that can unlock the true potential of WebAssembly.
In this tutorial we’ll discuss the ideas and concepts behind rendering water and then talk through some demo code.
Rust 2018 has shipped, and we’re closing in on the end of the year. While we didn’t manage to ship async/await as part of the edition itself, the community has made quite a lot of progress toward that goal. This post summarizes the state of play, and announces the publication of several crates intended to facilitate use of async/await on the nightly ecosystem.
Adjusting an existing Rust project to build a native binary and library for WASM, load in JS. Details on working through build issues.
Throw down your main! Rustlang Serverless HTTP applications won’t need them where they’re going
Take a developer deep dive into Terrarium, our multi-language, browser-based editor and deployment platform at the edge. Learn how to compile Rust programs to WebAssembly right on your local machine, interact with the Terrarium system, and explore some applications we’ve built with it.
I am elated to announce that the next major release of Rocket is now available! Rocket 0.4 is a step forward in every direction: it is packed with features and improvements that increase developer productivity, improve application security and robustness, provide new opportunities for extensibility, and deliver a renewed degree of toolchain stability.
This past weekend I made a game for Ludum Dare 43. Tools used: Aseprite, quicksilver. Inspired by Zachtronics. It is written in Rust and compiled to WebAssembly.
WebAssembly (abbreviated Wasm) is a binary instruction format for a stack-based virtual machine. This essentially means that is is fast, because the program is compiled to a much more compact format, making it faster to parse. Wasm can be written by hand if you're looking for a challenge, but is primarily meant to be written in another language, and then compiled to Wasm. You may know a little about Assembly language and how it works - here's a quick refresher in-case you're rusty.
Taking advantage of AWS Lambda Runtime support for Rust
Rust support on AWS Lambda was recently released, which seems like as good an opportunity as any to share some code and the solutions to challenges I encountered along the way ☺. I’ve decided to create a little diceware service, and the lambda-runtime crate provides a great API to make this a breeze.
This blog post is about a project called Romio that I’ve been working on over the past two or three weeks. Romio is a port of a small part of the Tokio project to the newer futures APIs.
I started the project to get some experience porting code from the old futures API to the new API. However, we realized that this code could also be useful to other people who want to experiment with networking code using the new async/await syntax, so with the help of others we polished it up during the RustFest Rome “impl days” and now its being released for people to experiment with.
Enjoy this quick introduction to creating a web server with Actix and monitoring it with Sentry.
We recently sent out a survey regarding the state of the current Rust web ecosystem and we got over a 1000 responses! We really appreciate the feedback from the community. This will help us continue to improve upon the state of the Rust web ecosystem. Today, we would like to go over the responses and understand the results.
AWS Lambda, which makes it easy for developers to run code for virtually any type of application or backend service with zero administration, has just announced the Runtime APIs. The Runtime APIs define an HTTP-based specification of the Lambda programming model which can be implemented in any programming language. To accompany the API launch, we have open sourced a runtime for the Rust language.
Since the last post on Tide, there have been a number of excellent contributions from a bunch of new contributors! In this post, I want to talk about the work that @tirr-c has done to substantially improve the middleware story.
In this article, we describe how we ported the Exonum framework to actix-web using generic programming.
Abusing Go runtime in AWS to run Rust binaries
I really like Elm. It is a delightful language with an amazing ecosystem. It has an interesting architecture called TEA, The Elm Architecture. Another language I like is Rust. On paper, Rust is completely different from Elm, but in using them both, I have seen some resemblance. Having used both Elm and Rust I had something I wanted to try. Would it be possible to create The Elm Architecture in Rust?
Say you want to store a huge number of very small filesthat you will only access over HTTP.For example:You are using rustdoc to render the documentation of a library.Without much work you’ll end up with about 100k HTML filesthat are about 10kB each.As it turns out,this number of small files is very annoying for any kind of file system performance.Best case: making copies/backups is slow.Worst case: You’re using an anti virus software and it takes ages.
After the positive response to the routing and extraction proposal in Tide, I’m pleased to say that an initial implementation is available on GitHub! As a testament to the strong foundation that Rust’s ecosystem provides, the basic framework implementation took only about 1,000 lines of code.
This is going to be the first post ( and hopefully not last ) in a series of posts about writing (and thinking about) serverless applications in Rust. Stay tuned…
First of all, hello there! This is the first announcement after the call for maintainers back in summer, and includes the efforts of several new faces: @whitfin, @nyarly, @colinbankier. Together, as well as input from the original authors @bradleybeddoes and @smangelsdorf, we are the new maintainers of the Gotham project. Today we’re excited to announce the release of Gotham 0.3, which follows the evolution of the broader Rust ecosystem.
Picking up from part one, we now have as server that takes an email address from a request and spits out a JSON response with an invitation object. In part one I said that we will send an email to the user, after some thought and feedback, we will be skipping this part now (look out for part 3). For now we will use the http response from the server to verify the email so to speak.
This is part four in a series of Rust on Azure Functions. The other parts are about performance comparisons (part 1 and part 2), and explaining the PMX algorithm (part 3). If you want to learn more about Genetic Algorithms be sure to read part 3 first).
When WebAssembly was first shipped it was an MVP which, while minimal, has spawned a huge number of exciting projects which work today across all major browsers. Rust has capitalized on the wasm MVP’s success as well with tools like wasm-bindgen and wasm-pack by making the MVP feel less minimal. WebAssembly is yet more ambitious, though! Since inception it’s always been intended to extend the WebAssembly specification with new features and functionality.
We are going to create a web-server in rust that only deals with user registration and authentication. I will be explaining the steps in each file as we go. The complete project code is here repo. Please take all this with a pinch of salt as I’m a still a noob to Rust.
In this post, I look at how WebAssembly can be used to create serverless functions and demonstrate an AWS Lambda function written entirely in Rust.
This post continues the series on Tide, sketching a possible design for routing and extraction that combines some of the best ideas from frameworks like Rocket, Actix, and Gotham.
The API is written in Rust, a language new to the Pi-hole project. Rust is a safe and fast language which matches well with our goals for the API. It is statically typed and prevents whole categories of errors while being productive and extendable.
Markdown rendering is very important to the performance of Semaphor - every message you send and read is a Markdown document - so we're always looking for ways to improve the performance of rendering Markdown. A couple months ago Jonathan Moore and I wondered how easy it would be to integrate WebAssembly into a React component, replacing the render() function, and we thought that moving Markdown parsing into Rust would be a great way to test this idea out.
WebAssembly is a binary instruction format for a stack-based virtual machine. Wasm is designed as a portable target for compilation of high-level languages like C/C++/Rust, enabling deployment on the web for client and server applications.
Rust has its goals set on to be a primary WASM language and it would be awesome to use it both in backend and frontend web. Ruukh is one of such efforts to realise that dream. Ruukh, a frontend web framework, is inspired by both VueJS and ReactJS.
This blog post is an update to the preceeding article “A web application completely written in Rust” and summarizes the projects’ progress over the last months.
I am proud to announce that I’m working on a book for the Pragmatic Programmers. The title hasn’t been set in stone yet, but the book will be about programming WebAssembly with Rust.
We are pleased to announce the first release of the web-sys crate! It provides raw bindings to all the Web’s APIs: everything from DOM manipulation to WebGL to Web Audio to timers to fetch and more!
Today we are releasing RedBPF and ingraind, our eBPF toolkit that integrates with StatsD and S3, to gather feedback, and see where others in the Rust community might take this framework. If you are looking to up your company’s monitoring game, gather more data about your Raspberry Pi cluster at home, or just have a strong academic interest in Rust and low-level bit shepherding, you might want to read on.
Today, Tower Web 0.3 has been released and it comes with two major new features: Experimental support for async/await. Support for using templates to render responses.
Smithy, a web development framework written in Rust that compiles to WebAssembly. In the mean time, there has been substantial progress, and Smithy is on the verge of being ready for alpha use! In this post, I want to describe the improvements that have been made, and what’s on the Smithy roadmap!
The 2.0 release of Linkerd brings two very significant changes. First, we’ve completely rewritten Linkerd to be orders of magnitude faster and smaller than Linkerd 1.x. Linkerd 2.0’s data plane is comprised of ultralight Rust proxies which consume around 10mb of RSS and have a p99 latency of <1ms. Linkerd’s minimalist control plane (written in Go) is similarly designed for speed and low resource footprint.
The Network Services Working Group aims to improve the story for web development this year in several respects: by bolstering foundations like async/await, by improving the ecosystem of web-related crates, and by pulling these pieces together into a framework and book called Tide. The name “Tide” refers to “a rising tide lifts all boats”; the intent is to improve sharing, compatibility, and improvements across all web development and frameworks in Rust.
Let’s take a look at how in Rust you can have an algorithm generic over T, where T is further bound by a trait, which itself is generic over one of several parameters.
tower-web version 0.2.2 has been released. It comes with a number of new features, which I will talk about in this post. Primarily, the middleware story is starting to come together. I will be expanding some on how middleware fits into Tower and web in general.
I set out out my goal 9 for Rustnish: Write benchmark code that compares runtime performance of Rustnish against Varnish. Use cargo bench to execute the benchmarks.
The basic idea of a performance test here is to send many HTTP requests to the web service (the reverse proxy in this case) and measure how fast the responses arrive back. Comparing the results from Rustnish and Varnish should give us an idea if our performance expectations are holding up.
In Servo, task-sources are implemented via a channel, whose sender is cloned for each specific task-source, and where tasks are messages sent on the channel and containing a closure representing the actual task.
In "Oxidizing sourmash: Python and FFI" I described my road to learn Rust, but something that I omitted was that around the same time the WebAssembly support in Rust started to look better and better and was a huge influence in my decision to learn Rust. Reimplementing the sourmash C++ extension in Rust and use the same codebase in the browser sounded very attractive, and now that it was working I started looking into how to use the WebAssembly target in Rust.
Today, let’s go through an entire fetch in Servo, starting with an example in JS.
Rust's web frameworks ecosystem is in constant change, but recently a new framework called warp came out implementing a new, original way to solve the old problem of transforming a request into a response, and I wanted to give it a try.
And, as I use GraphQL massively at work, I also wanted to check how well Juniper implements it. To add some spice, I used MongoDB as a storage engine instead of the ubiquitous and well-supported SQL databases.
A sensible, modern pastebin written in Rust
The short version is, Tower Web 0.2 was just released and regular Rust attributes are now used instead of magic comments. The doc comment is replaced with #[get("/")]. This is thanks to Rust macro wizard David Tolnay. I also thought that it would be best to immediately push out 0.2 and then we can all pretend 0.1 didn’t happen.
This is a demonstration of a pure-rust library to generate syntax diagrams for macro_rules!().Diagrams are generated fully automatically from rust-source as Scalable Vector Graphics, using customizable CSS for layout.
I previously announced Tower and mentioned that a web framework was in the works. It took longer than I had hoped (as it sometimes does with software), but today, I am opening up Tower Web.
Tower Web is an asynchronous HTTP web framework that focuses on removing boilerplate. It is built on top of Tokio, Hyper, and of course Tower. It works today on stable Rust.
In the first article of this series, we created a RestAPI for our application using Postgres, Rust, and Rocket. For the second half, we’ll be using React to create the front end.
Over the past several months, I’ve been working a web framework in Rust. I wanted to make use of the new hyper 0.12 changes, so the framework is just as fast, is asynchronous, and benefits from all the improvements found powering Linkerd. More importantly, I wanted there to be a reason for making a new framework; it couldn’t just be yet another framework with the only difference being I’ve written it. Instead, the way this framework is used is quite different than many that exist. In doing so, it expresses a strong opinion, which might not match your previous experiences, but I believe it manages to do something really special.
I’m super excited to reveal warp, a joint project with @carllerche.
For this post, I’m going to make three comparisons: to Flash, to Java Applets, and occasionally to PNaCL. Secondly, this post is going to focus on the web use-case for WebAssembly, even though the previous post was about non-web uses. We’ll make that comparison in the next post. Finally, this post is kind of like eating tapas, there’s a bunch of little sections.
A three day tour of Yew and WASM with Rust
Over the past few months, I’ve been writing Smithy, a very work-in-progress front-end WebAssembly framework written in Rust.
My goal for Smithy is to enable you to use idiomatic Rust to write front-end code. This has costs: for example worrying about lifetimes and using Rc
> to share state. But this also has the potential to give you the safety guarantees that the Rust compiler provides when writing browser code!
The idea was simple, choose a package and recursively traverse npm to find all of the packages that depend upon it, after all the npm website offers a ‘Dependents’ tab on a package’s page. I had not anticipated the issues that made this an interesting technical problem.
We've just finished music festival Atlas Weekend which took place in Kyiv, Ukraine. This year there were for about 450,000 visitors during 6 days. I'm glad to announce that Rust and actix-web are used as a backend for main technical purposes of festival.
In our last post about actix we introduced you to the Rust programming language and the actix actor framework. This week we will build a basic TCP client with actix.
My latest software architectural experiment is to write a complete real-world web application in Rust with as less as boilerplate as possible. Within this post I want to share my findings with you to answer the question on how much web Rust actually is.
The last blog post laid out the Rust and WebAssembly domain working group’s overall vision for Rust and WebAssembly. In this blog post, we will dive into the details of wasm-bindgen, the future we envision for it, and how you can help us build that future.
A tutorial on how to compile Rust to WebAssembly and integrate it into a React application
This is the first part in a two-part introductory series to PRRR Stack (Postgres, Rust, Rocket, React) application.
In our previous post, we saw how combining channels with an event-loop could be a useful technique to ‘drive’ the concurrent logic of your system, and it was hinted at that shared mutable state might be more complicated. A good example of such ‘shared mutable state’ in Servo is the HTTP cache.
I had this goal of making a web app that was very fast, stable and easy to deploy. This is what I was hoping to accomplish: strongly typed server-side and client-side languages (Rust & Elm respectively), push-button deployments to the cloud with free hosting initially, and sub-second response times for API calls and page loads. With this setup, I think I'm well on my way to accomplishing each of these.
Better HTTP Upgrades with hyper It’s been possible to handle HTTP Upgrades (like Websockets) in hyper if you made use of the low-level APIs in the server and client, but it wasn’t especially nice to...
This game was coded in Rust and is playable in web browsers by means of WebAssembly, WebGL and Howler.js. The software I developed is partly open source in the form of Gate, which is the Rust library that powers this game and can power other similar games. Special thanks to the tools I used to create assets: Gimp, FL Studio and BFXR.
Rust and WebAssembly can combine in many delightful ways. To consolidate our efforts and have the largest, most-positive impact we can, the Rust and WebAssembly domain working group is focusing on one vision:
This blog post will expand on these aspirations and describe where we stand in relation to them right now. In a series of follow up posts, we will talk about the next steps for each major component of the Rust and WebAssembly ecosystem.
Lately I’ve been working on a side project that involves aggregating the indexes of media libraries and allowing one library to “lend” a file to another. For example, I might use this system if I want my home tower to “lend” a file to my laptop before I get on a plane. I’ll be committing my work-in-progress on this project to github shortly (just want to get an MVP working first and do some refactoring), but I thought I would take a step back for a moment and share my thoughts on one aspect of this project: gRPC. In particular: gRPC streaming.
The 3rd of a multi-part series where we build a small REST client for the Marvel Comics Web API using Rust. This time we focus on how to use the hyper and futures crates to make HTTP requests to a Web API, then use serde to parse JSON data from the response.
Welcome to the third post in a series where I share my experience learning Rust. I’m building out a conspiracy theories API to help me get more familiar with Rust and have a little fun. Since I am new to Rust, I welcome all feedback, especially from developers who have been using Rust for quite some time. Leave a comment below or send your feedback via Twitter. Now it’s time to draw the blackout curtains, put on your foil-lined hat on and start serving up conspiracies from a REST API.
The “actor model” is the main primitive that powers the Erlang programming language and its descendant, Elixir. It describes a programming model that simplifies the development of concurrent and multi-threaded applications or even applications that run distributed on multiple machines.
The complexity of actors is relatively low, and that is because the complexity is usually hidden in the actor frameworks that are used to run these types of primitives in the end. One example of such an actor framework is actix, which we will have a closer look at now.
I was inspired by Nick’s post to migrate my code from Rocket to Actix. I have also been nagged by recent struggles with the nightly compiler and its compatibility with all the other crates that I use and other the platforms that I use; my code runs on 32⁄64 bit arm and 32⁄64 bit x86. When I started using Rocket around a year ago, the nightly compiler was just as stable as the stable compiler, but things have changed as rust is a growing eco system.
For me, once the differences below were addressed, porting the code was relatively straightforward. The vast majority of my code remained the same.
Creating a simple web service for automating Travis builds across repositories using actix-web and Github webhooks.
Today sees the release of hyper v0.12.0, a fast and correct HTTP library for the Rust language.
This release adds support for several new features, while taking the opportunity to fix some annoyances, and improve the extreme speeds!
While the http crate generally has a great API I have been unsatisfied how it handles URLs. To create a HTTP request a full URL is needed with a scheme (http/https), authority (example.org) and a path (/search?q=rust) but http does enforce this and allows you to only state the path. This means both clients and servers are either unable to determine protocol and and authority information or have to do this manually.
A little while ago I came across this Snowflake generator. It’s a project by Raph Levein that takes a hash string and uses it to procedurally generate a unique snowflake. He explains that the original motivation was as a cryptographically secure visual hash, so that people would reliably be able to tell by visual inspection whether two hashes were identical. I thought that was a pretty cool idea.
I write code for the web every day, and as such, I naturally wanted to find a web server that I could write some beautiful code in Rust with. There are a few options out there, Actix, Rocket, but none of them are quite as simple and succinct as KoaJS for node. As such, I wanted to write something as simple as Koa and as performant as a Rust framework, and I did. With that, I give you Thruster, an elegant, performant, web server written in Rust.
The 2nd of a multi-part series where we build a small REST client for the Marvel Comics Web API using Rust. This time we focus on how to use the hyper and futures crates to make HTTP requests to a Web API, then use serde to parse JSON data from the response.
I previously wrote an article back in November 2017: Replacing Elasticsearch with Rust and SQLite. In it, I needed to create a few HTTP endpoints that ingested JSON, perform a database lookup, and return JSON. Very simple. No query / path parameters, authentication, authorization, H2, or TLS. I didn’t understand how stability was such an important feature. I was familiar with needing new versions of the nightly compiler to stay current with clippy and rustfmt, but it was a blindspot when it came to dependencies.
Here I go with my first post that fully focuses on Rust. After spending a few months doing a bit here and there I decided to just dive right in as I was going through the Rust book at too slow a pace to keep myself interested. So, in this post I decided to write about setting up a simple REST API which is something that I have done in Java plenty of times but with Rust it is a different story.
I had the need for a very small API for this website. So small, in fact, that only one endpoint was required. I've been doing a lot of development in Rust lately, so naturally Rust seemed like a good candidate to build this API in. I also wanted to try out a newer Rust web framework called Actix web. It claims to be a "small, pragmatic, and extremely fast rust web framework", which sounded perfect for what I needed.
This is a small idea that I have been wanting to put together for quite some time now and finally have managed to get the time and most importantly experience in Rust to finally try something a little more than just small projects. One area that I think Rust is really making a decent headway in in the web domain, which I am assuming is likely due to its origin from Firefox. So I wanted to see if I could put together a really basic CRUD micro-service doing the ever so original TODO functionality.
Using Rust on Microsoft Azure Functions with web assembly
A project used for a meetup talk about getting started with rust & wasm
Hopefully this post helps jumpstart those who want to use LLVM from Rust. I also include how one could use this to generate WebAssembly, but the first part of this post is target platform agnostic.
C, C++, and Rust all have some capability for dynamic dispatch: function pointers, virtual methods, and trait objects. On native targets like x86, all these forms compile down into a jump to a dynamic address. What do these forms compile down into when targeting WebAssembly?
The Rust language is one of the earliest adopters of WebAssembly, and it has more than one way to compile to it:
wasm-pack is a tool for assembling and packaging Rust crates that target WebAssembly. These packages can be published to the npm Registry and used alongside other packages. This means you can use them side-by-side with JS and other packages, and in many kind of applications, be it a Node.js server side app, a client-side application bundled by Webpack, or any other sort of application that uses npm dependencies. You can find wasm-pack on crates.io and GitHub.
I wrote the NES emulator with Rust and WebAssembly to learn Rust. It’s not perfect and have some audio bugs, but it’s good enough to play Super Mario bros.
Rust aims to be the foundation of fast and robust software. A thorn in that side is the choice of timeout-centric APIs for I/O and multithreading. I posit this was a bad choice and it needs to be remedied as soon as possible before more APIs proliferate this (possibly accidental) design decision.
Yew is a new web framework written in Rust that takes concepts from React JS and Elm. It takes the two web libraries I love and combines them with the new language I love — Rust. I wrote this article because I haven’t found a lot of documentation online on how to get the Yew examples running, and I’m sure there will be people that will run into the same problems I did. I wrote this in hopes that it helps someone out there.
Serving content from a Rust web server running on a Raspberry Pi from your home to the world, with a Cloudflare Argo Tunnels.
Hello and welcome to the first issue of This Week in Rust and WebAssembly! Rust is a systems language pursuing the trifecta: safety, concurrency, and speed. WebAssembly is designed as a portable target for compilation of high-level languages like C, C++, and Rust, enabling deployment on the web for client and server applications. This is a weekly summary of its progress and community.
Tower is a library for writing robust network services with Rust. It is being built in service of the Conduit proxy, which is using the Tokio ecosystem to build the world’s smallest, fastest, most secure network proxy. Tower will also provide a batteries included experience for implementing HTTP and gRPC services.
Please read the Part 1 of the series here :
For one of our customers at Centricular we were working on a quite interesting project. Their use-case was basically to receive an as-high-as-possible number of audio RTP streams over UDP, transcode them, and then send them out via UDP again. Due to how GStreamer usually works, they were running into some performance issues.
This blog post will describe the first set of improvements that were implemented for this use-case, together with a minimal benchmark and the results. My colleague Mathieu will follow up with one or two other blog posts with the other improvements and a more full-featured benchmark.
The short version is that CPU usage decreased by about 65-75%, i.e. allowing 3-4x more streams with the same CPU usage. Also parallelization works better and usage of different CPU cores is more controllable, allowing for better scalability. And a fixed, but configurable number of threads is used, which is independent of the number of streams.
The newest release of hyper includes some lower-level connection APIs for both the server and client. Notably, this allows using hyper send and receive HTTP upgrade requests. The most popular of these is Websockets.
I started writing mob, an multi-echo server using mio, in 2015. I coded mob into a mostly working state and then left it mostly alone, only updating it to work with the latest stable mio. Recently, I started looking at the code again and had the urge to improve it. In a previous post, I talked about managing the state of connections in mob. In this post, I will walk through what I did to remove the need to track connection state. I wanted to remove the state because the implementation required an O(n) operation every tick of the mio event loop. It also added a fair amount of complexity to the code.
Blazing fast, low requirements, computationally intensive operations on Node.js using Rust
Central to the problem are the edges. People will reliably do a good job of building and testing the happy paths, but as humans we’re terrible at considering the edge conditions, and it’s those edges and corners that cause trouble over the years that a program is in service.
Since my initial Node/Rust REST comparison, I’ve wanted to follow up with a comprehensive guide for getting simple CRUD operations up and running in Rust.
A brief post to help others multicast in Rust
With the combined goal of gaining a deep understanding of DNS, of doing something interesting with Rust, and of scratching some of my own itches, I originally set out to implement my own DNS server. This document is not a truthful chronicle of that journey, but rather an idealized version of it, without all the detours I ended up taking. We'll gradually implement a full DNS server, starting from first principles.
WebAssembly has been interesting me lately, specifically the prospect of doing arithmetically-intensive operations with it, not unlike the fantastic physics engine Emscripten port Ammo.js. Compiling something like that is out of the scope of this little post (but should be getting easier!), and I’ve left some links at the bottom so you can clear more about WebAssembly if you’re interested.
I’m writing a reverse proxy in Rust using Hyper and I want to measure performance a bit to know if I’m doing something terribly wrong. By doing that I discovered a Denial of Service vulnerability in Hyper when IO errors are not properly handled. Note that a workaround has been released in the meantime in Hyper 0.11.20, more background info can be found in this Hyper issue.
IPFS is a peer-to-peer protocol that allows you to access and publish content in a decentralized fashion. It uses hashes to refer to files. Short of someone posting hashes on a website, discoverability of content is pretty low. In this article, we’re going to write a very simple crawler for IPFS.
It’s challenging to have a traditional search engine in IPFS because content rarely links to each other. But there is another way than just blindly following links like a traditional crawler.
I’ve been writing a small toy project with Tokio in my spare time’s spare time. I’ll write more about it at a later date. What I’ve found writing it, though, is that there is a specific pattern to writing servers, both UDP and TCP (and presumably others) in Tokio, and it’s not super obvious at first glance. So here it is.
Let me begin this article on Writing a Microservice in Rust by talking about C++. I’ve been a reasonably active member of the C++ community for quite a while now, attending and contributing talks to conferences, following the development and evangelism of the language’s more modern features and of course writing lots of it. C++ is a language that gives its users very fine-grained control over all aspects of the program they are writing, at the cost of a steep learning curve and a large body of knowledge required to write effective C++. However, C++ is also a very old language, conceived by Bjarne Stroustrup in 1985, and thus caries a lot of technical debt even into modern standards.
Vyacheslav Egorov, who goes by mraleph on the Web, wrote a response to my article “Oxidizing Source Maps with Rust and WebAssembly” titled “Maybe you don’t need Rust and WASM to speed up your JS”.
It’s been a few months since I shifted my focus full time to Rust, and in that time, we’ve gotten a lot of work done! I wanted to update you here what all that is, since it’s spread around on multiple repositories.
I recently spent some effort trying to make reproto run in a browser. Here I want to outline the problems I encountered and how I worked around them. I will also provide a number of suggestions for how things might be improved for future porters.
Fanta is a web framework that aims for developers to be productive and consistent across projects and teams. Its goals are to be: Opinionated, Fast, and Intuitive. Based heavily off of the work here: https://github.com/tokio-rs/tokio-minihttp
My nice brother Johannes Ridderstedt sent me some old files a few weeks ago (in late 2017), stuff that he had preserved from an age-old computer of ours. One of these was the file named gameland.zip (not published yet, but I might put it up here some day.) I managed to get this running, and liked what I saw (you'll find the YouTube link to it further down on this page.) Around this time I was reading a bit about WebAssembly which I think will redefine and help reshape the web as we see it today. I was also looking at the Hello, Rust web page, and the "FizzleFade effect using a Feistel network" page in particular.
Everyone loves event sourcing, right up until the moment they have to implement it. At that moment all the wonderful whiteboard drawings…
Introducing wee_alloc. wee_alloc is a work-in-progress memory allocator designed for WebAssembly. It has a tiny code size footprint, compiling down to only a kilobyte of .wasm code.
概要 以前、JSで書いた(ファミコンのエミュレータを書いた - undefined)ファミコンのエミュレータをRustで書き直してみた。 また、技術的な内容はQiitaの方にも書いているので興味のある方は参照してみてください。(まだ Hello, World!までしか書けてませんが。) qiita.com もともとファミコンのエミュレータって新しい言語を習得するのにちょうどいい題材だったりするのでは、って話しからスタートしてて、よくわからないのでJSで書いてみて、ようやくRustで一通りは実装できた感じ。まだバグや未実装（音声周りやマッパー）も多いんですが、ひとまずはお腹いっぱいな感じ。 成果…
DataFusion is an open-source Big Data platform implemented in the Rust programming language with a similar programming style to Apache Spark.
In this post, I'll walk through a few of the highlights of getting Turtle, a Rust library for creating animated drawings, to run in the browser with WebAssembly.