DevOps and Deployment

In this post we will set up a simple, serverless data ingestion pipeline using Rust, AWS Lambda and AWS SES with Workmail.

We will handle multiple types of AWS events with one Lambda function, parse received emails with the mailparse crate, and send email with SES and the lettre crate.

This post describes how Ebbflow vends its client which is written in Rust to its Linux users, describing the tools used to build the various packages for popular distributions. This guide starts off assuming we can build .deb and .rpm packages locally which we covered in our previous blog post, but now we want to move towards distribution. Specifically, we will discuss how the client is built for all of its target platforms using GitHub Actions. The Docker images and GitHub Actions that we discuss are all public and usable by anyone.

This post describes how Ebbflow vends its client to Windows users which is written in Rust, describing the tools used to build and ultimately deliver the program to users.

The Rust Infrastructure Team is happy to announce that, as part of the evaluation we started last year, most of Rust’s CI is moving to GitHub Actions! While we don’t expect the change to have any impact on our users, the switch will considerably improve the experience for compiler contributors.

Rust crates ecosystem has this amazing and thriving part, which is focused on linting, auditing, fuzzing and packaging other Rust projects: cargo-audit, cargo-geiger, cargo-release and so on. Obviously, it is very common to use them in the CI in order, for example, to fail builds automatically if changes introduced security vulnerabilities or new dependencies has the incompatible licenses.

There is a one small problem, though: most of these crates are your usual executable binaries, meaning that you need to compile them before use. While it is fine for a developer environment, adding cargo install line into your CI workflow impacts execution time drastically as it takes minutes to build the requested program and a couple seconds then to execute it.

The infrastructure team is happy to report that the evaluation we started last year of GitHub Actions as the new CI platform for the rust-lang/rust repository is making progress!

Lately I decided to give Rust microservices a try. I had already tinkered a lot with Flask or Express — and I still am perfectly happy to use them in most of the cases — but I felt like I needed to learn a new way of doing things to deal with critical microservices that need performances.

The main hurdle I came across while trying to develop such services was the lack of literature on how to handle a real DevOps workflow within Rust ecosystem. As you work here with compilers and binaries instead of interpreters, there are quite some changes to account for regarding the usual workflow. In this article, I’ll propose a workflow to develop efficiently locally, share your work through Gitlab, and automate code checks, test, and builds with Docker. I hope to make this into a series of article about how to deploy your app with Kubernetes on GCS and provision your infrastructure with tools such as Terraform.

If you are reading this then it’s quite likely you have discovered your love for Rust quite a while ago and are already somewhat proficient in using this amazing language.

I won’t boast with my rather lacking Rust-skills or try to provide another tutorial to get into this amazing beast of a language. There are tons of other great ressources to learn Rust, like. Rustlings or the Rust Book.

What I want to show you is another aspect of the Rust ecosystem that makes life so much easier.

Github Actions make a compelling option if one of your main challenges are to build binaries over wide array of platforms such as Linux, Windows, and macOS.

While the difference between a Linux and a given macOS in terms of build aren’t significant (we’ll see how they can become significant actually), Windows and especially how to get to a nice Rust build on Windows using MSVC require an investment on your part.

For the most part the Rust cross-compilation toolchain holds its weight well when you’re looking to build for various architectures, but we’ll see a few cases where you’re on your own with Github Actions.

Serverless containers and functions are widely used for deploying and managing software in the cloud. Their popularity is due to reduced cost of operations, improved utilization of hardware, and faster scaling than traditional deployment methods. The economics and scale of serverless applications demand that workloads from multiple customers run on the same hardware with minimal overhead, while preserving strong security and performance isolation. The traditional view is that there is a choice between virtualization with strong security and high overhead, and container technologies with weaker security and minimal overhead. This tradeoff is unacceptable to public infrastructure providers, who need both strong security and minimal overhead. To meet this need, we developed Fire-cracker, a new open source Virtual Machine Monitor (VMM)specialized for serverless workloads, but generally useful for containers, functions and other compute workloads within a reasonable set of constraints. We have deployed Firecracker in two publically available serverless compute services at Amazon Web Services (Lambda and Fargate), where it supports millions of production workloads, and trillions of requests per month. We describe how specializing for serverless in-formed the design of Firecracker, and what we learned from seamlessly migrating Lambda customers to Firecracker.

Distributing and packaging your Rust GUI application and making it available for Linux users can be hard, I will try to explain the various ways of doing that using Flatpak as a packaging format.

Kata Containers is an open source project and community working to build a standard implementation of lightweight Virtual Machines (VMs) that feel and perform like containers, but provide the workload isolation and security advantages of VMs.

In this post, we will look how AWS Lambda Functions can be implemented using Bastion. AWS Lambda functions are well known serverless platform used for tasks varying from infrastructure management, serving APIs, data processing to orchestration tasks.

In the previous articles, we discussed the scope of the container runtime shim and drafted the minimum viable version. Now, it's time to move on and have some fun with more advanced scenarios! Have you ever wondered how `docker run -i` or `kubectl run --stdin` work? If so, this article is for you! We will try to replicate this piece of functionality in our experimental container manager. And as you have probably guessed, the container runtime shim will do a lot of heavy lifting here again.

In my last blog post I said I wanted to spend some time learning new things. The first of those is Rust. I had previously tried learning it, but got distracted before I got very far.

Since one of the things I'd use Rust for is web pages, I decided to learn how to compile to WebAssembly, how to interface with Javascript, and how to use WebSockets. At home, I use a Mac to work on my web projects, so for Rust I am compiling a native server and a wasm client. But I also wanted to try running this on redblobgames.com, which is a Linux server. How should I compile to Linux? My first thought was to use my Linux machine at home. I can install the Rust compiler there and compile the server on that machine. Alternatively, I could use a virtual machine running Linux. Both of these options seemed slightly annoying.

In order to use runc from code we need to implement our shim as a daemon and this daemon has to be as long-lived as the underlying container process. In this article, we will try to develop a minimum viable runtime shim and integrate it with our experimental container manager.

The minimal shim implementation takes as its input a path to the container bundle (with the config.json) as well as the list of the predefined locations (for the container log file, container pidfile, container exit status file, etc). The shim needs to create a container by executing runc with the provided parameters and then serve the container process until its termination.

Until recently, Cargo didn’t support building crate’s dependencies independently of the binary/library, which made using a dummy crate the simplest way to take advantage of the Docker cache.

Fortunately, cargo vendor was created and solves just this issue: it allows you to vendor your dependencies into a local directory so that dependencies can be cached by Docker.

There are already several good articles on how to develop on Rust for Android. However, they describe how to build Rust manually (from the command line). That might be not the most convenient way to do it, since it makes the whole compilation and installing a several step process. I would prefer if Rust compilation was done automatically by gradle.

Crates are a good feature to use your code on different projects or share it to the world. But sometimes you just want to keep it private.

Since Cargo 1.34 was released, we are able to create private crates instead of the public ones that are published on crates.io. Here we are going to give a brief introduction of what you need to do to create, push and pull your first private crate using CloudSmith.

This is more or less the same for Go and the like where the binary(“exe”) can listen to on a given port. That and I want to thank Sutean for his post similar to well this one; however, I wanted to dig in a little further into the application pool explanation and provide some additional insight for those that only have to setup IIS every blue moon having my own documented. Further I’m going to be extremely basic so something that seems straight forward will be included. I also used Actix web for my server, but again this could be any framework where the executable can listen on a port.

This blog post is a slightly edited version of the live transcript of the talk I gave at RustFest 2019 in Barcelona on November 10th, 2019. As it’s a transcript some parts of it are a bit repetitive or badly worded, but I hope the message behind the talk will be conveyed by this post anyway.

The Rust Infrastructure team is happy to announce that we’re starting an evaluation of GitHub Actions as a replacement for Azure Pipelines as the CI provider of the rust-lang/rust repository.

We can make a lot of things with Rust like web apps, system drivers and much more but there is one problem, which is the time that Rust takes to make a binary by downloading dependencies and compile them.

We need to make a pipeline using the Gitlab CI/CD and Docker to make the deployment faster.

At Meili, we needed a tool that allows us to monitor our pods, we already have vigil which health checks our front page and backend, but the number of these services is limited. We do not pop new front or backend servers dynamically (for now). When we create new search engines for the user we instanciate a kubernetes pod, we need to monitor the health of this service. Adding each of those URLs by hand in the vigil config file is not a solution.

So we decided that we needed a simple tool, a tool that can accept HTTP requests to register/unregister URLs to health check. We use the new async/await Rust syntax along with tide for the http server, no big deal here.

Last week I finally released the first version of espanso, a cross-platform, system-wide Text Expander written in Rust. I’ve built many projects in my life, but this time I wanted to create something people would actually use. Therefore, one of the key aspects to consider was the installation process: it has to be easy. Being a cross-platform tool, I had to consider the best installation method for each platform and when it comes to macOS, the de-facto way to install a cli tool is by using Homebrew. In this article I will show you the publishing process I used for my tool, espanso.

A few weeks ago I got beta access to the GitHub CI/CD platform called Actions. For my own Rust projects I was using Travis CI mostly before and had started migrating to the Azure Pipelines recently, but now I’m considering moving to the Actions instead.

Configuration syntax is a bit nicer comparing to Azure one, there are Linux, macOS and Windows environments available (pretty much as everywhere now) and integration with other GitHub parts is just amazing. But the most important thing there for me is a possibility to create “Actions” — custom tasks to be executed in the CI workflow.

I've been building a program to track how I spend my time, and a couple of the requirements are an interface on my phone, and sync between multiple devices. Since I haven't built an android app in forever, I decided to target termux, at least initially. This will allow me to use the app and test out the synchronization protocol without a ton of upfront effort into building a graphical user interface.

Part of our stack at Clevy is written in Rust, and we use Neon to ease the bindings with other parts of the stack written in NodeJS. Recently, we needed to deploy this stack on AWS Lambda, which runs a very specific NodeJS runtime, not cross-compatible with our existing stack. Since we struggled a little bit with getting Lambda/Rust/Node to play nicely together, I figured I would post a short how-to of what we found worked well for us. You can of course take this as a base and change it to your liking!

It’s been about a month since we released kube, a new rust client library for kubernetes. We covered the initial release, but it was full of naive optimism and uncertainty. Would the generic setup work with native objects? How far would it extend? Non-standard objects? Patch handling? Event handling? Surely, it’d be a fools errand to write an entire client library?

In the Go world, it is common to use docker’s multi-stage build feature to produce the app’s docker image in two stages. The first stage uses the golang image and is where we build the application into a statically-linked binary. Once built, we copy the binary into a scratch container in the second stage. The result is a rather small image that has nothing other than the application.

I was looking to do the same thing for my Rust application. While I was at it, I also wanted to leverage the docker build cache to avoid having to download crates on every docker build.

If you are working in a team that’s writing Rust code and your crates have a good amount of dependencies, you have probably noticed that the Rust compilation phase takes a non-trivial amount of time in comparison to actually running your tests: it is indeed one of the top recurring complains that gets reported to the language team.

In a CI environment, it gets worse as the testing container starts building pretty much from scratch and everything gets recompiled. One way to alleviate that problem is to cache Rust compiled intermediate objects

In this article, I’m going to show you how to fetch private Cargo dependencies and source them when building a Docker image. This solves a key issue with Docker of not copying over SSH keys when building an image.

Writing light weight cloud services without Go.

Recently I wrote a binary called "Watchdog" in Rust on my Mac that I wanted to take with me to Linux systems I frequent, but I couldn't bring a Rust compiler to these systems for unrelated reasons. This meant I had to cross-compile my application and ship just the binary.

For a little side project I’m working on I want to be able to produce pre-compiled binaries for a variety of platforms, including FreeBSD. With a bit of trial and error I have been able to successfully build working FreeBSD binaries from a Docker container, without using (slow) emulation/virtual machines. This post describes how it works and how to add it to your own Rust project.

In this article, we’ll explore Apache OpenWhisk (which we’ll refer to simply as OpenWhisk), an open-source, distributed serverless platform that allows us to execute functions in response to events. OpenWhisk supports a lot of languages out of the box and can be extended to use other languages. In our case, we are going to use Rust as our main primary language.

How does one package a Rust app? I would check out creating a snap package (https://t.co/UYibrbPN7c). It worked alright for me, even though the format encourages sandboxing of apps which takes a bit to wrap your mind a bit. Or create a deb using cargo-deb and use the alien utility to transform that to a rpm.

In this post I will detail why I believe that Azure Pipelines can be a great CI / CD platform for open source Rust projects on Github. The catch is that there are some rough spots on Azure Pipelines and in the rust ecosystem, but everything can be worked around. In writing this post, I hope to detail examples one can copy and paste into their projects.

A refreshing new story for running Rust on AWS Lambda.

A few days ago, we discussed how to write AWS Lambdas in Rust. Today, we’re going to learn how to create and deploy an Azure Function using Rust and the azure-functions-sdk library.

At babylon health we have a ton of microservices running on kubernetes that are, in turn, controlled by hundreds of thousands of lines of autogenerated yaml. So for our own sanity, we built shipcat - a standardisation tool (powered by rust-lang and serde) to control the declarative format and lifecycle of every microservice.

Hello everyone! Today I am excited to announce that DocQL is open-sourcing a Rust crate (a library) which we built internally called…

I cover first impressions on GitHub actions using Rust as well as how to make it do a simple merge with a comment!

In the fall of 2017, we decided to write Firecracker in Rust, a modern programming language that guarantees thread and memory safety and prevents buffer overflows and many other types of memory safety errors that can lead to security vulnerabilities. Read more details about the features and architecture of the Firecracker VMM at Firecracker Design.

Firecracker microVMs improve efficiency and utilization with a low memory overhead of < 5 MiB per microVMs. This means that you can pack thousands of microVMs onto a single machine. You can use an in-process rate limiter to control, with fine granularity, how network and storage resources are shared, even across thousands of microVMs. All hardware compute resources can be safely oversubscribed, to maximize the number of workloads that can run on a host.

Abusing Go runtime in AWS to run Rust binaries

How to setup test a Rust Nightly project with a workspace in Travis CI and collect coverage information with kcov and Codecov.

Learn how to cache your Docker builds with Rust better so you don't keep downloading and building your deps over and over and over again!

K∅RQ is used for tailing pod logs concurrently and following groups at once. It was basically a need to follow logs during deployment and see how instances behave during and after deployment. This is the main motive behind K∅RQ.

Today, I was working on optimizing my travis builds to speed them up, especially in the direction of caching. Travis allows you to set cache: cargo in your .travis.yml to enable caching for rust projects and caching is cool: it avoids having to rebuild all dependencies all the time, speeding up the builds.

OpenFaaS is an open source implementation of Function as a Service (Serverless Functions, microservices) that you can self host. Rather than list all the various offerings in this space, I'll refer you to the Cloud Native Computing Foundation, in particular the interactive Landscape.

You can either deploy existing functions or create new ones. If you create new ones, there's a big list of officially supported languages. Alternative you could turn a CLI into function.

Once I'd given Python and Ruby a go as an introduction, I wanted to see how easy it would be to create a Rust template.

We are excited to announce the next version of vagga, the containerization tool that makes setting up development environments easy and fun. This release brings local overrides for vagga commands, support of ubuntu bionic, better CI support, space reuse between multiple projects, and more.

Using a tiny Rust app to demonstrate deploying Rust with Docker and Kubernetes.

This is just a small build script to automatically compile and create Rust binaries for your i86 32 and 64 bit machine and your Raspberry Pi(ARM) with Travis-CI. On the right side I added a little explanation of the following script.

Containers are moving the world. Each passing day more and more organizations are embracing containers as first-class citizens for distribution and deployment of software components. Containers represent the core of the cloud native paradigm.

Here is a small walkthrough, on how to use Mac OS X, Linux or even Windows (hey not tested but I am sure you can make it work, the tools are the same) to compile your Rust marvellousness and run the binary directly on the Raspberry Pi (2/3/3+).

Today we’re very happy to announce Conduit 0.5.0, which introduces zero-config automatic TLS between mesh’d pods (including certificate creation and distribution). This means that most Kubernetes users can now encrypt internal HTTP communication between their service in just two simple commands.

We’re also happy to announce that 0.5.0 will be the last major release of Conduit. Conduit is graduating into the Linkerd project to become the basis of Linkerd 2.0. Read on for what this means!

I had this goal of making a web app that was very fast, stable and easy to deploy. This is what I was hoping to accomplish: strongly typed server-side and client-side languages (Rust & Elm respectively), push-button deployments to the cloud with free hosting initially, and sub-second response times for API calls and page loads. With this setup, I think I'm well on my way to accomplishing each of these.

tl;dr - I applied a few patterns I’ve used on other projects to a Gitlab CI-powered rust project to achieve <2min builds. Basically just caching at different layers – caching via the docker image builder pattern at the docker level, aggressive caching with Gitlab CI at the CI runner level, also one more step of combining some build steps (probably unnecessarily).

We're going to create a multi-stage pipeline based on the one I'm using at work, featuring: a build image, linting, test & release builds, and docker containers

Since I started at Microsoft about 6 months ago, I had to catch up on their (our) tech: Azure, .NET, Visual Studio. Yet as an avid GitHub user I overlooked one thing completely: Visual Studio Team Services! Turns out it’s quite the hidden gem and after a colleague showed me some of its power, I had to look into it more deeply. Turns out it’s an easy and free resource for any project. Can it help your project? Yes! How? Find out below 😊

On June 13, 2017 took place the Paris Container Day. They unveiled a new docker feature: multi-stage build. That's the subject of this article.

Lets deploy small docker images for Rust