It’s time to make sup, our own take on ping, use the Win32 APIs to send an ICMP echo. Earlier we discovered that Windows’s ping.exe used IcmpSendEcho2Ex. But for our purposes, the simpler IcmpSendEcho will do just fine.
As we mentioned earlier, it’s provided by IPHLPAPI.dll, and its C declaration is:
IPHLPAPI_DLL_LINKAGE DWORD IcmpSendEcho( HANDLE IcmpHandle, IPAddr DestinationAddress, LPVOID RequestData, WORD RequestSize, PIP_OPTION_INFORMATION RequestOptions, LPVOID ReplyBuffer, DWORD ReplySize, DWORD Timeout ); Compared to MessageBoxA, there’s a lot more types going on!
Tools and Applications
Command line tools and GUI applications built with Rust or built for Rust.
When talking to various people at conferences in the last year or at conferences, a recurring topic was that they believed that the GTK Rust bindings are not ready for use yet.
I don’t know where that perception comes from but if it was true, there wouldn’t have been applications like Fractal, Podcasts or Shortwave using GTK from Rust, or I wouldn’t be able to do a workshop about desktop application development in Rust with GTK and GStreamer at the Linux Application Summit in Barcelona this Friday (code can be found here already) or earlier this year at GUADEC.
Today, I am about to guide you in the depths of crates.io and how I made an alternative search bar using our instant search engine: MeiliDB.
We are proud to present Mun v0.1.0 - our first milestone release. As a language, Mun is still far from production-ready, but this release gives you a glimpse of what natively supported hot reloading will look like in the future. The purpose of this release is to showcase our progress and gather feedback from those brave souls willing to try out Mun at this early stage.
Shortwave is an internet radio player that lets you search for stations, listen to them and record songs automatically.
ptail is a small, and likely useless, utility that truncates the output from processes. Unlike tail -f it will not show more than the specified number of lines in your terminal. This could be useful if you are executing a command as part of a shell script and you do not wish to show the full and verbose output, but you do want to be able to see what is happening.
In the last few weeks, I've been working on a new solution to firmware management on the Linux desktop. A generic framework which combines fwupd and system76-firmware; with a GTK frontend library and application; that is written in Rust.
I've been writing GTK applications in Rust for a few years now. I've also been in a unique position with a career at System76, where my day job relies on writing software exclusively for Linux in Rust, including GTK widgets and applications. I'm now at a point where I'm comfortable sharing what I've learned, and therefore this post will explain some of the best practices, patterns, and crates that I use today in my day to day job, from the perspective of a Rust-based software developer that had no prior experience in GTK before Rust.
Yet another HackerNews CLI written in Rust.
I'm finally happy to announce that I've "finished" the 1.29 branch of mrustc (for those who don't know, mrustc is my attempt at making a rust compiler, primarily for breaking the bootstrap chain).
This version is capable of compiling both rustc 1.29 (and packaged cargo) AND rustc 1.19, both of which can compile their successors (1.20 and 1.30 - and 1.30 results in a binary equal output).
In this post we are starting building a logger using Rust as a programming language and Scylla DB as a data storage. This post is about data layer which is responsible for writing data to and reading it from a database.
A simple command-line tool to generate application icons from raster graphics or vector graphics images. Quickly combine multiple images and create professional-looking icons for most versions of Windows, macOS, iOS & Linux. Powered by IconBaker.
I got a project idea to test the feasibility of implementing Spark in a native language and if feasible, explore how efficient it can be in terms of performance and resource management. I know that Spark is heavily optimized over the years. I didn’t hope for any drastic difference in performance and if some difference is there, it most likely will be in RAM usage. Also, I want it to very general-purpose just like Spark. I decided to use Rust for the implementation.
Bayard is a full-text search and indexing server written in Rust built on top of Tantivy that implements The Raft Consensus Algorithm (raft-rs) and The gRPC (grpc-rs). Achieves consensus across all the nodes, ensures every change made to the system is made to a quorum of nodes. Bayard makes easy for programmers to develop search applications with advanced features and high availability.
what sniffs a given network interface and records IP packet size, cross referencing it with the /proc filesystem. It is responsive to the terminal window size, displaying less info if there is no room for it. It will also attempt to resolve ips to their host name in the background using reverse DNS on a best effort basis.
espanso detects when you type a keyword and replaces it while you're typing.
logq is my latest project and it has reached a reasonably qualitative milestone for me to comfortably introduce it and share about its technical detail.
Let’s start by an example. Imagine that you are in the middle of troubleshooting a production incident and you are suspecting a certain endpoint from the web-server is having problem and result into high latency. Since it is application level and it is not provided by the AWS CloudWatch. It is also the first time that it happened so there isn’t any in-house datadog or application level instarumentation setup. And it occurs to you that the access log contains the relevant information and it would be possible for you to calculate the metrics from the log. You download the log from the archive storage and piece together an ad-hoc script in 30 minutes and run the metrics against the log, and the script is implemented in python it gets to pretty slow if the log size is large. Wouldn’t it be great if there were command line where you could handle these kind of ad-hoc analysis situation easily? Where no extra dependency setup like ELK or library is needed. That is the motivation to drive to the development of logq, where you could answer the question of “What are the 95th latencies with 5 seconds time frame against application handlers by ignoring the second path segments” easily.
This post showcases a small CLI tool written in Rust using both synchronous and asynchronous execution flow.
A story about lack of optimisation and the weaknesses of the human mind.
Have you ever wanted to try a new crate in Rust Playground, then you realized sadly that crate hasn’t made to the top 100 most downloaded crates on crates.io? Now you have to go through the process of creating a new Cargo project (cargo new playground) and editing Cargo.toml (vim Cargo.toml) to add that crate and write the code (vim src/main.rs) and opening your terminal to run the code with cargo run. So many commands to run to just try a new crate. Wouldn’t the world be nicer if you can just open your favorite editor to write some code and a command will take care of all these hassles for you? Like a local Rust Playground?
Meet cargo-play, a brand new Cargo custom subcommand that brings Rust Playground to your computer and give you full control on what crates you can play with.
We've just released Nushell 0.5.0! New - login shell support, recycle bin support, better string matching, parameter help, new commands, and more!
End of the year, meaning it's time for a new GNOME+Rust hackfest and its feedback!
Last week, I went to the sixth Rust+GNOME hackfest which was in Rome. During these hackfests, we work on improving the integration between Rust and the GNOME libraries.
Nushell, or Nu for short, is a new shell that takes a modern, structured approach to your commandline. It works seamlessly with the data from your filesystem, operating system, and a growing number of file formats to make it easy to build powerful commandline pipelines.
Today we’re happy to announce the 0.4.0 release of Nushell. The 0.4.0 release marks a continually maturing shell that is now starting to show signs of stability. And, of course, a few fun features along the way.
The idea to create Mun originated out of frustration with the Lua dynamic scripting language that is extensively used for game development at Abbey Games.
Lua's hot reloading capabilities and LuaJIT's performance make it a great language for rapid prototyping of real-time applications - such as games - on PC. However, the language has performance issues on some mobile and console platforms - to which LuaJIT cannot deploy, the language lacks refactoring functionality, and does not scale well with modern technology.
Mun tries to take the best of both worlds to create a more robust, highly iterative, productive, and performant programming language.
Spotify TUI: Spotify for the terminal written in Rust.
Nushell, or Nu for short, is a new shell that takes a modern, structured approach to your commandline. It works seamlessly with the data from your filesystem, operating system, and a growing number of file formats to make it easy to build powerful commandline pipelines. We’re happy to announce that today we’re releasing Nushell 0.3.0. Nu has seen numerous bugfixes, performance improvements, and features added since its initial public release (which was only a few weeks ago!)
Currently the only implementation of crev is cargo-crev, which ties into the Rust language package manager, cargo. However, none of this is Rust-specific apart from the implementation, the basic concept and code review format should work for any language or package system. Code reviews (“proofs”) are just YAML files, and they can be shared around however you feel like – the method currently seems to be by putting proofs in git repositories, and cargo-crev has support for this. crev already has a pretty good getting started guide that covers much of the same ground as this, but I wanted to write something similar that comes from a random user, not the system’s creator.
Our goal is to build useful tools that make it easier to take existing Rust code and get up and running with Rust. We aim to automate much of the translation and rewriting process so that migrating legacy systems is practical and scalable with minimal manual effort.
We’re excited to announce that a milestone long in the making is finally here! You can now install C2Rust from crates.io with a simple cargo install on Linux and OS X. We’ve been hard at work improving C2Rust, so go install and give it a spin! You can find the necessary prerequisites in the C2Rust README.
Today, we’re introducing a new shell, written in Rust. It draws inspiration from the classic Unix philosophy of pipelines, the structured data approach of PowerShell, functional programming, systems programming, and more.
rx is an extensible, modern and minimalist pixel editor implemented in Rust. It's designed to have as little UI as possible, and instead takes inspiration from vi's modal nature and command mode. Compared to other pixel editors, rx aims to be smaller, yet more configurable and extendable. `rx` takes a different approach when it comes to animation as well, which is done with *strips*.
pastel is a command-line tool to generate, analyze, convert and manipulate colors. It supports many different color formats and color spaces like RGB, HSL, CIELAB, CIELCh as well as ANSI 8-bit and 24-bit representations.
It’s a double challenge: I’m learning Rust (but I can say I more or less know the language by now), and I’m trying to use GTK in Rust. I’m a complete beginner in GTK, and even if I know Rust, it does not mean I can use it properly. So I’ll learn a huge library (written in C) by using it in a complex and hard language I just learned. Well, well, good luck, me.
An early release with basic features and only Maildir support has been published in meli’s git repositories. meli is a new experimental mail client for the terminal. It’s a from-scratch implementation in order to experiment with ideas I had about a client’s design.
The web platform is the delivery mechanism of choice for a ton of software these days, either through the web browser itself or through Electron, but that doesn’t mean there isn’t a place for a good old fashioned straight-up desktop application in the picture.
Fortunately, it’s easier than ever to write a usable, pretty, and performant desktop app, using my language of choice (Rust) and the wildly successful cross-platform GUI framework GTK. In this blog post, we’ll walk through the source code of gDiceRoller. In future posts, I’ll explain how I packaged it for different systems.
Search the code of all crates published to crates.io.
Web tool to evaluate rust regular expressions.
Today, I'm excited to announce the first release of PyOxidizer (project, documentation), an open source utility that aims to solve the Python application distribution problem! (The installation instructions are in the docs.) PyOxidizer's marquee feature is that it can produce a single file executable containing a fully-featured Python interpreter, its extensions, standard library, and your application's modules and resources.
A Rust regular expression editor & tester inspired by Rubular.
rga is a line-oriented search tool that allows you to look for a regex in a multitude of file types. rga wraps the awesome ripgrep and enables it to search in pdf, docx, sqlite, jpg, zip, tar.*, movie subtitles (mkv, mp4), etc.
The vision is to build a livecoding / design hybrid program, where procedural design and code are fused in one environment. If you have missed 'learnable programming' please check this out: http://worrydream.com/LearnableProgramming/ Makepad aims to fulfill (some) of these ideas using a completely from-scratch renderstack built on the GPU and Rust. It will be like an IDE meets a vector designtool, and had offspring. Direct manipulation of the vectors modifies the code, the code modifies the vectors.
However before we can make this awesome application, we need to build a UI stack. The aim of this toolkit is to be our stepping stone into building a livecoding IDE and designtools that don't suck or fall to pieces along the way.
A long time ago I wrote a blog post about how to maintain a Flatpak repository. It is still a nice, mostly up to date, description of how Flatpak repositories work. However, it doesn’t really have a great answer to the issue called syncing updates in the post. In other words, it really is more about how to maintain a repository on one machine.
To support this I’ve been working on a side project called flat-manager. It is a service written in rust that manages Flatpak repositories. Recently we migrated Flathub to use it, and its seems to work quite well.
Emu is a language for programming GPUs from Rust. Emu provides the emu! macro for compiling functions written in the Emu language to an intermediate code which gets stored in the EMU global constant. You can then run your code using a binding to OpenCL such as ocl or rust-opencl or use the build! macro which generates Rust functions that can be called to run your Emu functions.
dua (-> Disk Usage Analyzer) is a tool to conveniently learn about the usage of disk space of a given directory. It's parallel by default and will max out your SSD, providing relevant information as fast as possible.
An interpreted language written in Rust inspired by the Lisp family of languages.
battop is an interactive viewer, similar to top, htop and other *top utilities, but about batteries installed in your notebook.
A curation of opinions and facts on a variety of Rust IDEs.
After open sourcing Snips-NLU a year ago, Snips now shares Tract, a new piece of its embedded voice platform. Tract is Snips’ neural network inference engine.
Why I created a command-line JSON processor.
A CLI tool for Linux that allows you to copy a partition from one disk to another and more. A library that allows managing GUID partition tables. Features include: Read/Write GPT from 512 and 4096 bytes sector size disks, Create a new GPT on a disk, Insert/delete a partition in the table, Align partitions automatically, Resize a partition, Copy/clone a partition from one disk and insert it to another, Change partition type, Fix partitions order, Change disk GUID, Change partition name, Change partition GUID, Toggle legacy BIOS bootable, Toggle no block IO protocol, Toggle required partition flag, Toggle attributes, Customize columns to print, Print raw data of disklabel, Swap partition indexes, Randomize disk's GUID and all partition's GUID, Copy/clone all partitions from one disk and insert it to another.
Rust Playground for MacOS is a standalone native mac application that allows quickly editing and testing rust snippets.
Unusually for me, I had a bit of a pause in reviewing Sled. As a reminder, Sled is an embedded database engine written in Rust. I last stopped looking at the buffer management, but I still don’t really have a good grasp of what is going on. The next file is the iterator. It looks like it translates between segments and messages in these segments.
During the last few months at Videolabs, I added support for tile encoding in rav1e (a Rust AV1 Encoder). AV1 is an open and royalty-free video coding format, concurrent with HEVC (H.265). Rav1e is an encoder written in Rust, developped by Mozilla/Xiph. As such, it takes an input video and encodes it to produce a valid AV1 bitstream. Tile encoding consists in splitting video frames into tiles that can be encoded and decoded independently in parallel (to use several CPUs), at the cost of a small loss in compression efficiency. This speeds up encoding and increases decoding frame rate.
XV is a terminal hex viewer with a text user interface, written in 100% safe Rust. Licensed under the Apache-2.0 software license. Get it from crates.io by running cargo install xv in your terminal.
In my very slow and not very steady quest to learn the Rust programming language, I’ve come across a few projects written in the language that I use everyday. I thought I’d write a quick post about them, with some configuration tricks that I’ve made to make them suit my needs.
Why prefer tools built with Rust? Rust is a very fast language that also works to ensure safety from a group of bugs and pitfalls. It’s also the four-time-running most loved programming language, according to a yearly Stack Overflow survey. Also, since Rust is only a few years old, any program written in Rust is by definition new. Of course newer doesn’t always mean better, but as a non-professional developer I can afford to be on the edge a bit.
I always end up using Elasticsearch to index documents, to generate autocompletes and for geolocation. Sonic doesn’t solve all three problems but it is a good tool to solve the first two. I have not yet used it in production, but it seems like a good lightweight alternative to Elasticsearch.
Since we love databases and we are trying to focus on Rust projects, Amin Arria and I decided to interview Sonic’s creator, Valerian Saliou, who generously agreed.
Sled is an embedded database engine written in Rust. It takes a very different approach for how to store data, which I’m really excited to see. And with that, let’s be about it. In stopped in my last post when getting to the flusher, which simply sleep and call flush on the iobufs.
A tiling window manager written in Rust for stability and performance. The core of left is designed to do one thing and one thing well. Be a window manager. Because you probably want more than just a black screen LeftWM is built around the concept of theming. With themes you can choose between different bar / compositor / background / colors, whatever makes you happy. LeftWM has been built from the very beginning to support multiple screens and has been built around ultrawide monitors. You will see this with the default key bindings
In a previous post I described the current implementation of cargo-call-stack, a static stack usage analysis tool. In the second part of that post I described the problems the tool runs into when dealing with indirect function calls (both function pointer calls and dynamic dispatch) and proposed improving the output of the tool by having rustc inject type information in the LLVM IR it produces.
While discussing that idea with rustc developers they noted that the compiler could emit information that’s more relevant to call graph analysis than just the signatures of functions and trait methods. In this post I’ll describe the call graph metadata I’m currently implementing in rustc and how it will help tools like cargo-call-stack. By the way, I have a feature branch of cargo-call-stack that uses this upcoming rustc feature if you want to try it out but you’ll need to build a custom rustc.
Because keeping documentation up-to-date is very important (and should be a must have!), we need to check if the examples are still valid after every new updates. Luckily, rustdoc already makes such things very easy. But now, let's say you have examples outside of your Rust code. How can they be tested?
Alacritty is a terminal emulator with a strong focus on simplicity and performance. With such a strong focus on performance, included features are carefully considered and you can always expect Alacritty to be blazingly fast. By making sane choices for defaults, Alacritty requires no additional setup. However, it does allow configuration of many aspects of the terminal. With this release, Alacritty has officially entered the beta stage – there are still a few missing features and bugs to be fixed, but it is already used by many as a daily driver.
hunter is a fast and lag-free file browser/manager for the terminal. It features a heavily asynchronous and multi-threaded design and all disk IO happens off the main thread in a non-blocking fashion, so that hunter will always stay responsive, even under heavy load on a slow spinning rust disk, even with all the previews enabled.
The Sled project is an embedded database written in Rust. I run into it a few times recently and given my day job, I decided to take a peek and understand how it works. The project talks about being Log Structure Merge (and also exposing this to the client) with B+Tree read performance. The last time I read an LSM codebase was quite some time ago, so this is going to be quite interesting, I hope.
Concurrent and safe object-oriented programming, without the headaches.
As I’ve been experimenting with writing Rust apps, I attempted to create a small little GUI application. At first I attempted to setup everything with Qt, but C++/Qt interoperability with Rust is painful. Very, very painful. I experimented with some more radical UI frameworks such as Azul and Conrad. These have a lot of promise going forward. However for the here and now, I recommend looking at gtk-rs, Rust binding for Gtk.
cargo sync-readme synchronizes your README (the file specified by the readme key in your Cargo.toml, or just README.md by default) with the entrypoint of your library or binary crate (by default, lib.rs or main.rs, or what is defined at the path key in your manifest).
cargo-instruments (github) is a cargo plugin that makes it easy to profile rust binaries on macOS. tl;dr: cargo-instruments is a shim between cargo and Xcode's very powerful dtrace-backed diagnostic suite, Instruments. Out of the box, this lets you track cpu / thread usage, allocations, context switches, and a bunch of other stuff.
While Python is a great language to quickly build new features in, it’s not great for performance or deploying to multiple platforms (especially Android). So @johan-bjareholt took on the tremendous task of reimplementing aw-server in the highly performant and secure programming language Rust. Since Rust compiles to Android we have built a library for use on Android, letting us avoid reimplement all of ActivityWatch to Android devices specifically (and vastly decreases the time to port future features like sync to Android).
Most in our team use CLion for Rust development which is especially great for local debugging, alas it is not free. Since we are developing a blockchain it requires careful orchestration of the nodes running on separate machines, and occasionally we need to debug some corner case on a remotely running node. CLion and other JetBrains products have great support of the remote development and debugging. However, unfortunately, Rust is not a primary language of CLion which makes configuration tricky. In this post we walk through the configuration of CLion for remote Rust development and debugging.
In this post we will finally have some music. I will spend some time discussing how the Amiga sound hardware worked. The original mod file format and how it is played back is intimately linked with the Amiga hardware.
rust-vmm facilitates sharing core virtualization components between Rust Virtual Machine Monitors.
Torchbear is a new programming environment currently available for Windows, Android, MacOS, and Linux. With it, you get Rust's great library ecosystem, but you don't need to recompile with each change to how you use those libraries. Since you express your domain logic in a higher level scripting language and your system logic in Rust's language, you get the best of both worlds!
I finished my last post with having a fully parsed mod file in memory and ready to be played. By the end of this post I want to get the code into a state where the main thread lets the user pick instruments from the mod file and play them back on the audio thread.
I’ve written quite a bit about the theory of patches and merging, but nothing yet about how to actually implement anything efficiently. That will be the subject of this post, and probably some future posts too.
Almost two years ago, I promised a series of three posts about version control. The first two (here and here) introduced a new (at the time) framework for version control. The third post, which I never finished, was going to talk about the datastructures and algorithms used in pijul, a version control system built around that new framework. The problem is that pijul is a complex piece of software, and so I had lots of trouble wrapping my head around it.
Two years later, I’m finally ready to continue with this series of posts (but having learned from my earlier mistakes, I’m not going to predict the total number of posts ahead of time). In the meantime, I’ve written my own toy version control system (VCS) to help me understand what’s going on. It’s called ojo, and it’s extremely primitive: to start with, it can only track a single file. However, it is (just barely) sophisticated enough to demonstrate the important ideas. I’m also doing my best to make the code is clear and well-documented.
Many people have had the idea that there should be a way to visualize lifetimes in Rust. Indeed, the Rust Book used to include ASCII diagrams of lifetimes in some code examples. When fighting the borrow checker, it would be great if the IDE or editor could automatically provide a visualization of the lifetimes in your code.
My last article finished with reading the pattern tables from the mode files. In this post I want to finish parsing the entire file so we can move onto playing it. First we need to work out where the pattern data is and how long it is.
For my next Rust project I want to try something a bit more challenging than the Sudoku solver. I want to write a mod player in Rust.
Check out git branches by their pull (or merge) request ID
After the recent breach, I was curious to check my passwords against the list, but I’m a bit paranoid, so, rather than paste my passwords into the Have I Been Pwned website, I wanted to download the big text file and check my passwords against it offline, nice and safely.
I use a password manager called KeePassXC, so all of my passwords are stored in an encrypted file – a KeePass database – and I use a program called KeePassXC, a free and open-source password manager, to manage them (I wrote a beginner’s user guide to KeePassXC a while back if you’re interested!). So ideally, to check my passwords against the big list, I’d have a tool that checks all the passwords in a given KeePass database against the entire HaveIBeenPwned list of passwords, preferably against the downloaded file (i.e. “offline”), rather than the API. In other words something similar to 1Password’s Watchtower feature, but preferably offline.
After poking around a bit I decided to write it myself in Rust, with this script and this crate as useful references. Medic is a Rust CLI that can perform a variety of “health” checks on a KeePass database.
I like to find ways to fill up my free time over winter breaks. Since I’m not in school for a month, I usually take the time to do some self-study on a topic I find interesting. This past month, it was building a database. Unfortunately, I had only a casual understanding of how a database actually works under the hood, and no idea how to get started. Additionally, I decided to write the database in Rust, since I knew there would be complicated management of data, and I wanted to avoid headaches involving invalid data. Finally, rust’s static garbage collector meant I wouldn’t have to rely on a runtime to guarantee this.
That was a month ago. While I’m not too far along (mostly due to being busier than expected during the break), I still have things I think are worth sharing.
A while back I started thinking about software and how it does what it does. Perhaps even more importantly, I started thinking about how we can know what software does, without relying on the software's marketing pitch.
Although GoReleaser supports building just Go projects, it does so much more in the packaging and distribution department that it is extremely hard to ignore.
In my previous post about Python to Rust transpiler I’ve said, that one of the biggest problems is absence of types in python and inability to infer them for functions. But it turns out there is a solution to this problem. Guys from Instagram have developed an incredible tool for runtime type inference called MonkeyType. It monitors what types functions accept and return while running a program. This approach works quite nicely so I want to share with you how it’s done.
I am pleased to announce that the first release of sandboxfs, 0.1.0, is finally here! You can download the sources and prebuilt binaries from the 0.1.0 release page and you can read the installation instructions for more details.
The journey to this first release has been a long one. sandboxfs was first conceived over two years ago, was first announced in August 2017, showed its first promising results in April 2018, and has been undergoing a rewrite from Go to Rust.
Lately I’ve been needing to reverse engineer some apks and I’ve been manually using some third-party tools to do so. But, how could we speed up the process? A little bit of Rust can be very helpful!
Goals: Full Python-3 environment entirely in Rust (not CPython bindings). A clean implementation without compatibility hacks
This post aims to give you a small introduction to using the Rust programming language for writing apps for Ubuntu touch.
Reliable: Ensure everyone in your project uses an identical dev environment, from the Node version to your preferred package manager.
Universal: Enjoy the same command-line experience in every shell and every major operating system.
Fast: Notion is implemented in Rust and deployed as a static executable for maximum performance.
Building a cross platform game for desktop operating systems in Rust is fairly doable without needing much platform specific code. Glutin is a Rust alternative to SDL for handling window creation & input. GFX handles most of the graphics API abstraction for you. You still write the shaders, but I was able to just use OpenGL and get it working on Windows 10, MacOS, and Ubuntu.
I recently wrote a plugin in Rust to control the Spotify desktop app for MacOS and find lyrics from within Neovim and found it to be a neat experience overall, and that’s coming from someone who isn’t a Rust expert. This post is a summary of all the research and discovery that happened during that process.
hexyl is a simple hex viewer for the terminal. It uses a colored output to distinguish different categories of bytes (NULL bytes, printable ASCII characters, ASCII whitespace characters, other ASCII characters and non-ASCII).
The usual process of playing something on our music server is a bit cumbersome: I have to browse SoundCloud, download a track that seems interesting, upload it to the music server, let MPD index it, and then play it.
To make my life easier, I wanted to leverage Linux’s FUSE interface to expose the audio on SoundCloud as a bunch of files in a folder which MPD could then index and play from. I could have taken a couple of other approaches such as swapping MPD with Mopidy, an MPD-compatible music player with SoundCloud support, or a scraper that periodically downloads the latest audio from my feed. And while these alternatives were probably easier to implement, I went for the FUSE driver because I thought it was cool.
PyOxidizer is a collection of Rust crates that facilitate building libraries and binaries containing Python interpreters. PyOxidizer is capable of producing a single file executable - with all dependencies statically linked and all resources (like .pyc files) embedded in the executable.
The following is a set of notes for installing VSCode on FreeBSD and getting a debugger up and running in a step by step guide. I thought I would share them in a full post as having a full IDE with syntax and error highlighting along with detailed in-context explanations has been really useful as I get to grips with Rust.
dutree is a command line tool to analyze disk usage.
Hi! I am Aleksey Kladov (aka @matklad). In the past, I've worked at Jet Brains where I've helped to create the IntelliJ Rust plug-in, and now I am a part of the Ferrous Systems team.
I've spent a significant amount of the last year experimenting with various approaches to make the Rust IDE story better. The culmination of my experiments is the rust-analyzer project – an experimental Rust compiler frontend, targeting the IDE/Language Server Protocol use case.
In this blog post I'll cover Clippy and Rustfmt – two tools that have been around for a few years and are now stable and ready for general use. I'll also cover IDE support – a key workflow for many users which is now much better supported. I'll start by talking about Rustfix, a new tool which was central to our edition migration plans.
Rust allows for a lot of syntactic sugar, that makes it a pleasure to write. It is sometimes hard, however, to look behind the curtain and see what the compiler is really doing with our code. I wondered if there was a tool, which revealed what Rust was doing behind the curtains.
This is a tutorial on building your own shell using Rust, in the spirit of the build-your-own-x list. Creating a shell is a great way to understand how the shell, terminal emulator, and OS work together.
This blog shows how Qt applications can be built with Cargo. The goal is to make compiling them as simple as installing Qt and running cargo build.
K∅RQ is used for tailing pod logs concurrently and following groups at once. It was basically a need to follow logs during deployment and see how instances behave during and after deployment. This is the main motive behind K∅RQ.
This is the first call for participation for the imag project. I have no experience writing such calls for participation, so please bear with me!
Right now, the imag ecosystem has some tools available which are already usable and in rather good shape. There is a contact manager, a diary and a notes tool, a habit tracker and a time tracker are there as well, though those are not extensively tested by now.
A few days ago, I accepted a new challenge of creating a simple command line barcode reader using Rust and Dynamsoft Barcode Reader SDK. Rust is a system programming language similar to C++. The learning process did not go through smoothly as I expected. It is not as easy as learning other high-level programming languages such as Java and Python. In this article, I share my experience of learning and using Rust.
Today I’d like to talk about a command-line tool I’ve been working on. It’s called ruplacer and as the name suggest, it’s rually cool and written in Rust. Basically, it finds and replaces text in source files.
As you doubtless know, Emacs has an embedded Lisp environment that provides a large number of intersting Lisp functions that the user can call. Many of these are implemented in C for speed, and we've been rewriting them in Rust. So far we've ported 394 individual Lisp functions from C to Rust, of which 207 were ported in this last year. This is about a third of the total, as you can see by this graph. We've actually completely ported several whole C files now.
runner is mostly a clever wrapper around normal Cargo operations. runner acts like an interpreter. There is no forced directory structure, just source. But it’s just using rustc under the hood in the most direct way possible.
Hegemon is a work-in-progress modular system monitor written in safe Rust. Currently, it has the following features: Monitor CPU and memory usage, temperatures, and fan speeds; expand any data stream to reveal a more detailed graph and additional information, adjustable update interval.
For a small side project I’m working on, I’m using a Sudoku puzzle solver and puzzle generator that I’ve written in Rust. The experience was fun, so I thought I’d write up a little bit about the algorithm I’ve used and some interesting stats about how it performs.
a totp generator I wrote a while ago in C++ but now it’s in rust. It can generate a token and save it into an account file that is AES encrypted. The password is never saved, so it’s secure enough to use it. One of it’s properties it will have over the c++ implementation is it’s safe, it uses a proper IV and once I’m done, it will also support encryption via PGP.
Let’s write a mail viewer with Rust and Qt. This is another blog about Rust Qt Binding Generator, the project that lets you add a Qt GUI to your Rust code, or if you will, add Rust to your Qt program.
Alacritty, the OpenGL terminal emulator written in Rust, now supports scrollback! Performance has improved, and we've got benchmarks to share.
At Datalust we build a log server called Seq. It's really a database, written in C#, with its own SQL-based query language, that's specifically designed for storing and querying structured log data. In the past, Seq has managed log event storage through ESENT, which is a storage technology baked into Windows. For our cross-platform Seq 5 release we’ve built a storage engine called Flare in the Rust programming language to replace our usage of ESENT.
Writing the same password generator in two different languages to learn more Rust.
In this series of short blog posts, we are going to take a closer look at the key Rust related features present in JetBrains IDEs through our corresponding plugin.
The c2rust project exists to help bridge the fact that there is a lot of valuable software written in C and that there have been great strides in making safer and more-reliable programming languages since C was designed. Rust offers many modern improvements for C while still preserving the low-level control that makes it attractive. Beyond that, Rust provides new abstraction capabilities like parametricity, type-traits, methods, a module system, thread-safety, and more.
I spent some days writing a portage eclass for Gentoo. I want to share my experience.
NLnet Labs is embarking on a new adventure. In the coming months we will be developing an RPKI toolset aimed at making BGP routing more secure. It will consist of three parts: a Certificate Authority package, a Publication Server and Relying Party software. We’ve chosen to do the development in a modern systems programming language, Rust.
With the latest release of Amp (text editor), syntax highlighting has been overhauled. This post dives into the performance-related redesign that has landed with v0.5.
Ever had some code base that you regularly use to start a new project? Until now, you've probably lost some time refactoring everything to fill out the right project name, title etc. A few tools already exist but either you're lost in feature bloating hell or you're fighting to configure everything in most cases. The following tool is an early-developement, rust-based, template tool made to be more versatile that its closest python counter-part, cookiecutter.
Don’t waste time waiting for your bundler to do its thing. Use Pax while you’re developing, and iterate to your heart’s content. Use your super-cool, magical, slow-as-molasses bundler for releases, when you don’t care how long it takes to run.
glitchcat is a cat-like program with glitch animation.
So, you collect metrics. So do we. Yep, we also collect them. We can assure you that they are not useless at all, business guys do need…
CLion 2018.2 was released just recently. Along with the release, the Rust plugin has had an update! In this blog post, we’d like to highlight some of the main improvements in the plugin.
Gutenberg 0.4.0 is out with custom taxonomies, image processing, improved shortcodes and more.
mazon S3 is a storage solution used by pretty much everyone these days. Due to this there are naturally a bunch of tools for doing almost everything you can think of with S3. This post is about a tool I wrote to retrieve metadata about S3 buckets. If you don't care for the post, feel free to skip straight to the repo.
This past fall, three former GnuPG developers began working on a new OpenPGP implementation in Rust called Sequoia. As it’s starting to shape up and become useful, I feel now is a good time to announce the project to the larger Rust community, and hopefully get some feedback before our first release.
In this post, we’re going to build a sequence-based recommender system in Rust: a system that accepts a person’s reading history as input, and outputs recommendations on what to read next.
The piano arrangement album 帰るべき城 by Altneuland was published in 2005. I discovered it in 2008 (probably on YouTube), downloaded the best copy I could find, and filed it away in the TODO list. Recent advances in international parcel forwarding technology let me buy a used copy last year, but when it arrived none of my CD drives could read track #3. This sort of thing is common when buying used CDs, especially if they need to transit a USPS international shipping center. I shelved it and kept on the lookout for another copy, which I located last month. It arrived on Friday, I immediately tried to rip it, and hit the exact same error. This didn’t seem to be an issue of wear or damage…
Lots of people asked me to write another piece about the internals of well-known Unix commands. Well, actually, nobody asked me, but it makes for a good intro. I'm sure you’ve read the previous parts about yes and ls — they are awesome.
Anyway, today we talk about cat, which is used to concatenate files - or, more commonly, abused to print a file's contents to the screen.
We are excited to announce the next version of vagga, the containerization tool that makes setting up development environments easy and fun. This release brings local overrides for vagga commands, support of ubuntu bionic, better CI support, space reuse between multiple projects, and more.
cargo-generate is a developer tool to help you get up and running quickly with a new Rust project by leveraging a pre-existing git repository as a template.
We recently had the opportunity to make a Visual Studio Code extension that needed to communicate with an embedded device. This post explains the architecture we chose to achieve that and the decisions that led to it.
Almost every project I’ve worked on has grown a shell script named “build.sh”, and not much later a “test.sh” and “run.sh”. At this point, you have to make a decision as a developer whether your goal is to accidentally reinvent make or if your codebase’s needs are better met by an executable to manage your workflow.
Nannou is an open source creative coding framework built in Rust for artists, hackers, designers and devs to express themselves with simple, fast, reliable, portable code.
As operator I found that existing tooling fails with large number of unusual devices. If you have server with >100 virtual machines, and each virtual machines have one or more additional disk and a few network interfaces, it’s hard to find which VM cause stress on the server. Normal atop simply couldn’t cope with hundreds of tap/tun interfaces, multitude of block devices, etc. When I found that I have no proper tooling (as an operator) I wrote a shabby tools to do my job: they are calling blktop and ifstop. Both were written dirty and fast, and they does not follow the usual Python conventions (but they work as standalone binaries). They do their job, and normally I would say that’s enough. But those utilities are simple and have no big complications under the hood, so they are perfect candidates for rewriting in Rust for the sake of writing in Rust (I’m learning it!). I decide to combine them into a single utility: devtop, which I want to develop properly, according to Rust conventions, and through all aspects of packaging.
It's like SSH, but more secure, and with cool modern features. It is not an implementation of SSH, it is a new, modern protocol.
Finda lets you search and instantly switch between tabs in browsers like Firefox and Chrome and editors like Sublime Text and Visual Studio Code. It does this by communicating with these programs over local websocket connections. This article describes the design tradeoffs of this system, focusing on Rust implementation details that may be helpful to others building similar systems.
I am currently working on Wayk Now written in C for the most part. We recently started writing new code for it in Rust which I’m super excited about.
As we needed a way to integrate this code in our CMake build1, we wrote a set of modules for CMake
I am quite notorious for exploiting Gitlab’s CI. Ever since I started playing with it at the start of 2016, I tended to make things worse.
The C2Rust project is being developed by Galois and Immunant. This tool is able to translate most C modules into semantically equivalent Rust code. These modules are intended to be compiled in isolation in order to produce compatible object files. We are developing several tools that help transform the initial Rust sources into idiomatic Rust.
In this post, I will walk you through configuring an EMACS Rust IDE to get syntax highlighting and code completion for the Rust programming language.
Last week I tweeted "What do you think are the most interesting/exciting projects using Rust? (No self-promotion :-) )". The response was awesome! Jonathan Turner suggested I write up the responses as a blog post, and here we are.
Rust Qt Binding Generator lets you combine Rust code with a Qt1 graphical application. A previous blog shows how to make a simple clock. It’s a good idea to read that post before reading this more advanced post, because in this post we are getting serious.
This blog post shows how to write a to-do application. The data model is a list of to-do items. The source code for this example is available in the folder examples/todos in the Rust Qt Binding Generator repository.
Ahead of this year’s World Password Day, 1Password – maker of password management software – announced a password cracking challenge. The company ostensibly wanted to find out how hard it would be to crack a three-word passphrase master password on one of their vaults, assuming that the attacker had the derived hash of the passphrase.
Back in Februari 2013 then coworker Romain S. showed me the new trend of programming editors that do continuous compilation while you type, showing you immediate feedback on your code. In parallel I also worked on 3D modeling for my 3D printer using the OpenSCAD program. OpenSCAD works by writing code in its custom language and then have it rendered. I had this idea of combining these two approaches to make an electronics footprint generator. And so the development of the original madparts program started. In August 2016 I had been playing with the then pretty new rust programming language and decided a rewrite in it and simplifying the program even further would be fun to do.
Small tutorial of using Smithay's Client Toolkit by example, building a small image viewer as a Wayland client.
dutree is a command line tool to analyze disk usage. Features coloured output, according to the LS_COLORS environment variable, display the file system tree, ability to aggregate small files, ability to exclude files or directories, ability to compare different directories fast, and written in Rust. This tool is a mix between tree and durep.
For a fun project, I’ve been tinkering with xi-win, an experimental Windows front-end for xi-editor, written in Rust. I’m basically optimizing for performance, so making a number of somewhat unusual decisions. Among other things, I’m writing the UI myself, rather than using an existing toolkit or framework.
sudo is used by engineers daily to run commands as privileged users. But on some sensitive systems, you really want to ensure that no individual can act entirely autonomously. At Square, this includes applications that manage our internal access-control systems, store accounting ledgers, or even move around real money. This plugin allows us to ensure that no user can act entirely on their own authority within these systems.
I’ve been thinking about information theory, entropy, and passphrases for a couple of months now. I’ve been particularly interested in using random passphrases as passwords. An example of one of these passphrases would be “stamina turret backlands ruby”. The words have to be as purely random as possible – using your four dogs’ names is not nearly as strong as a password, as an attacker would likely guess that relatively early.
Whilst sort -u is super easy to use, the requirement of sorting your data cannot be overlooked as it means your entire file needs to be buffered into memory at once. This is not always possible, or even if possible it's not always desired. On the other hand uniq is great, but in order to sort your data in advance you still have the same issue. It was due to this that I wrote runiq, which is essentially an optimized and more flexible implementation of both of the aforemention tools for the purposes of duplicate filtering. It performs much faster, with much less memory, and does not require sorted input (although it can optimize this case).
TL;DR seiri is a lightweight music manager written (mostly) in Rust that helps keep huge music libraries organized. Try it today at chyyran.moe/seiri.
This is a web-based Kanban board application, built with Elm and Rust. The only different from this application and hundred thousands of web-based application out there is: We don't need Electron!. Instead, it use native WebView (WebKit for Linux/macOS, and MSHTML on Windows),
In 2012, Mozilla started the Servo project, a community effort to create new open source browser components that can take advantage of multicore hardware to improve speed, stability and responsiveness. Servo is comparable to WebKit, an open source browser engine that is the foundation of Apple’s Safari browser. (Other browser engines include Blink, EdgeHTML, and Gecko.) The difference is that Servo is written in the Rust programming language, bringing lightning-fast performance and memory safety to browser internals. Rust code can drastically cut down the number of critical bugs that affect browser engines. Servo uses a modular architecture and currently runs on Linux, macOS, Windows, and Android operating systems.
bat supports syntax highlighting for a large number of programming and markup languages. bat also communicates with git to show modifications with respect to the index (see left side bar).
Way back in the dim mists of history (back in university) I wrote myself a custom RSS reader in Java and called it JARVIS1. You see, I read a lot of webcomics. Like, a lot. Some webcomics provide RSS feeds, but some don’t, and as my collection grew it started to become a hassle to use Firefox’s live bookmarks to manage it all. Ultimately, I wrote up a quick Swing GUI to use as a single interface for keeping up with blogs and tracking which comics had published updates since the last time I’d checked2.
Hammond has been in development since Sep 2017. It makes use of the Rust bindings to GNOME libraries as well as diesel, serde, reqwest, and a number other crates to build a beautiful, native application that feels right at home on the GNOME desktop.
fastmod is a fast partial replacement for codemod. Like codemod, it is a tool to assist you with large-scale codebase refactors, and it supports most of codemod's options. fastmod's major philosophical difference from codemod is that it is focused on improving the use case "I want to use interactive mode to make sure my regex is correct, and then I want to apply the regex everywhere". For this use case, it offers much better performance than codemod
I’m pleased to announce the new release of Pijul, version 0.10. Pijul is an easy to use, distributed and fast version control system. This release has been a long time coming, but brings in an important number of new features and stability enhancements.
cargo src is a new tool for exploring your Rust code. It is a cargo plugin which runs locally and lets you navigate your project in a web browser. It has syntax highlighting, jump to definition, type on hover, semantic search, find uses, find impls, and more.
Dust is meant to give you an instant overview of which directories are using disk space without requiring sort or head. Dust will print a maximum of 1 'Did not have permissions message'. Dust will list the 15 biggest sub directories or files and will smartly recurse down the tree to find the larger ones. There is no need for a '-d' flag or a '-h' flag. The largest sub directory will have its size shown in red
Angle-grinder allows you to parse, aggregate, sum, average, percentile, and sort your data. You can see it, live-updating, in your terminal. Angle grinder is designed for when, for whatever reason, you don't have your data in graphite/honeycomb/kibana/sumologic/splunk/etc. but still want to be able to do sophisticated analytics.
Angle grinder can process about a million rows per second, so it's usable for fairly meaty aggregation. The results will live update in your terminal as data is processed. Angle grinder is a bare bones functional programming language coupled with a pretty terminal UI.
A few months ago, Bünz, Bootle, Boneh, Poelstra, Wuille, and Maxwell published Bulletproofs, which dramatically improves proof performance both in terms of proof size and verification time. In addition, it allows proving a much wider class of statements than just range proofs.
At Chain, we (Henry de Valence, Cathie Yun and Oleg Andreev) have been working on a pure-Rust Bulletproofs implementation, whose initial version we are publishing today, together with a set of notes.
A software library for high performance and hardware accelerated simulation of Quantum Computers and Algorithms. Written with Rust and OpenCL.
This new version comes with great performance improvements. We're talking about 3x faster on macos, 2x faster on linux and 3x faster on windows (the benchmarks are at the end of the post).
Sequoia is a modular OpenPGP implementation in Rust.
pfr is a command-line tool for helping me to manage my personal finances. At the end of each month, I like to sit down and “allocate” funds to cover my expenses for the following month. However, not all of my expenses occur monthly. For example, things like rent are paid weekly, and I think of money spent on food in terms of weeks, so I have to multiply those expenses by 4 (ish). I also have to figure out how much money to put in each account, e.g on my EFTPOS (direct-debit) card or for automatic payments. It’s not hard, but then again, that’s why we write programs.
Click is an open-source tool that lets you quickly and easily run commands against Kubernetes resources, without copy/pasting all the time, and that easily integrates into your existing command line workflows. This blog introduces its its usage and merits.
I wish I could have the features of both tools, but for now I will use gutenberg over cobalt. A tiny migration and feature comparison story.
(or how rust helped me to match the speed of GNU join written by Mike Haertel)
Xray - An experimental next-generation Electron-based text editor. This update covers a revised roadmap, fast file finding, and thoughts on key bindings.
In case you have tried to make a rust program with GUI that would work both on Linux and Windows, you probably may have faced some issues in some point (especially if you are a newcomer). Compiling in Linux is quite easy, but in Windows life gets a bit tricky.
I've pulled together some info from here and there so that no-one else had to fight their way through fire and flames.
A tool for serving and managing presentation slides.
When I built Finda, I wanted it to be fast — specifically, to respond to all user input within 16 milliseconds.
Given this goal, you might be surprised to learn that Finda is built with Electron, a framework that’s often decried for being the opposite of fast.
In my series of useless Unix tools rewritten in Rust, today I'm going to be covering one of my all-time favorites: ls.
In recent years, it has become increasingly important to develop software that minimizes security vulnerabilities. Memory management bugs are a common cause of these vulnerabilities. To that end, the Mozilla community has spent the last several years building the Rust language and ecosystem which focuses primarily on eliminating those bugs. And Rust is available in Fedora today, along with a few applications in Fedora 27 and higher, as seen below.
A bit later than anticipated, this is now part two of the blog post series about writing GStreamer elements in Rust. Part one can be found here, and I’ll assume that everything written there is known already. In this part, a raw audio sine wave source element is going to be written. It will be similar to the one Mathieu was writing in his blog post about writing such a GStreamer element in Python. Various details will be different though, but more about that later.
The team at Paris-based Snips has created a voice assistant that can be embedded in a single device or used in a home network to control lights, thermostat, music, and more. You can build a home hub on a Raspberry Pi and ask it for a weather report, to play your favorite song, or to brew up a double espresso. Manufacturers like Keecker are adding Snips’ technology to products like multimedia home robots. And Snips works closely with leaders across the value chain, like NVIDIA, EBV, and Analog Devices, in order to voice-enable an increasingly wider range of device types, from speakers to home automation systems to cars.
spectra is a crate I’ve been maintaining for a few months / years now. It’s a crate that I mainly use for demoscene productions (I released two with it, Céleri Rémoulade and Outline Invitation) but I also use it to play around and experiment new rendering, animation and video game techniques.
Fanta is a web framework that aims for developers to be productive and consistent across projects and teams. Its goals are to be: Opinionated, Fast, and Intuitive. Based heavily off of the work here: https://github.com/tokio-rs/tokio-minihttp
comm is a peer-to-peer instant messaging protocol designed to be resilient to censorship. comm-gtk is a GUI client built on the comm library. To try it out, start the app. In the configuration window, enter a secret phrase, a bootstrap node (IP:port pair), and a local port to listen on (e.g. 6669). For a bootstrap node, try 18.104.22.168:6667 (or any other node's IP address if you know one). Click connect to join the network.
A fast command line client for tldr: A collection of simplified and community-driven man pages.
I had an itch: I was pretty-printing the BERT-encoded terms that we use in a production system at work and it was very slow. The Erlang shell took more than two minutes to dump the largest file. (It took about 0.1 second to read and parse the file; the rest was spent in io:format.) I decided to scratch that itch: I wrote ppbert, a command-line utility that reads BERT-encoded values and pretty-prints them. I’ve worked sporadically on ppbert for almost a year now, I use it daily at work, I’m happy with it, and I want to write about some of the things I learned during that journey.
In an effort to experiment with dsp, I wrote a guitar/bass effects processor this past weekend. The end result works very well (to my pleasant surprise). It doesn’t have 90% of the functionalities of any of rakarrack, guitar rig, garage band but overall it was a fun weekend hack.
DataFusion is an open-source Big Data platform implemented in the Rust programming language with a similar programming style to Apache Spark.
retrobasic is a BASIC interpreter written in Rust. It is based on the original BASIC '64 implementation, and modified as needed to make it more compatible with some of the classic BASIC games.
As a library writer, it feels a bit strange, but refreshing, to write a program that actually has a main() function. My experience with Rust so far has been threefold:
An obvious idea is to use Rust for task automation. Originally, I have proposed creating a special Cargo subcommand to execute build tasks, implemented as Rust programs, in this thread. However, since then I realized that there are built-in tools in Cargo which allow one to get a pretty ergonomic solution. Namely, the combination of workspaces, aliases and ability to define binaries seems to do the trick.
Cat for markdown: Show markdown documents in TTYs