K∅RQ is used for tailing pod logs concurrently and following groups at once. It was basically a need to follow logs during deployment and see how instances behave during and after deployment. This is the main motive behind K∅RQ.
Tools and Applications
Command line tools and GUI applications built with Rust or built for Rust.
This is the first call for participation for the imag project. I have no experience writing such calls for participation, so please bear with me!
Right now, the imag ecosystem has some tools available which are already usable and in rather good shape. There is a contact manager, a diary and a notes tool, a habit tracker and a time tracker are there as well, though those are not extensively tested by now.
A few days ago, I accepted a new challenge of creating a simple command line barcode reader using Rust and Dynamsoft Barcode Reader SDK. Rust is a system programming language similar to C++. The learning process did not go through smoothly as I expected. It is not as easy as learning other high-level programming languages such as Java and Python. In this article, I share my experience of learning and using Rust.
Today I’d like to talk about a command-line tool I’ve been working on. It’s called ruplacer and as the name suggest, it’s rually cool and written in Rust. Basically, it finds and replaces text in source files.
As you doubtless know, Emacs has an embedded Lisp environment that provides a large number of intersting Lisp functions that the user can call. Many of these are implemented in C for speed, and we've been rewriting them in Rust. So far we've ported 394 individual Lisp functions from C to Rust, of which 207 were ported in this last year. This is about a third of the total, as you can see by this graph. We've actually completely ported several whole C files now.
runner is mostly a clever wrapper around normal Cargo operations. runner acts like an interpreter. There is no forced directory structure, just source. But it’s just using rustc under the hood in the most direct way possible.
Hegemon is a work-in-progress modular system monitor written in safe Rust. Currently, it has the following features: Monitor CPU and memory usage, temperatures, and fan speeds; expand any data stream to reveal a more detailed graph and additional information, adjustable update interval.
For a small side project I’m working on, I’m using a Sudoku puzzle solver and puzzle generator that I’ve written in Rust. The experience was fun, so I thought I’d write up a little bit about the algorithm I’ve used and some interesting stats about how it performs.
a totp generator I wrote a while ago in C++ but now it’s in rust. It can generate a token and save it into an account file that is AES encrypted. The password is never saved, so it’s secure enough to use it. One of it’s properties it will have over the c++ implementation is it’s safe, it uses a proper IV and once I’m done, it will also support encryption via PGP.
Let’s write a mail viewer with Rust and Qt. This is another blog about Rust Qt Binding Generator, the project that lets you add a Qt GUI to your Rust code, or if you will, add Rust to your Qt program.
Alacritty, the OpenGL terminal emulator written in Rust, now supports scrollback! Performance has improved, and we've got benchmarks to share.
At Datalust we build a log server called Seq. It's really a database, written in C#, with its own SQL-based query language, that's specifically designed for storing and querying structured log data. In the past, Seq has managed log event storage through ESENT, which is a storage technology baked into Windows. For our cross-platform Seq 5 release we’ve built a storage engine called Flare in the Rust programming language to replace our usage of ESENT.
Writing the same password generator in two different languages to learn more Rust.
In this series of short blog posts, we are going to take a closer look at the key Rust related features present in JetBrains IDEs through our corresponding plugin.
The c2rust project exists to help bridge the fact that there is a lot of valuable software written in C and that there have been great strides in making safer and more-reliable programming languages since C was designed. Rust offers many modern improvements for C while still preserving the low-level control that makes it attractive. Beyond that, Rust provides new abstraction capabilities like parametricity, type-traits, methods, a module system, thread-safety, and more.
I spent some days writing a portage eclass for Gentoo. I want to share my experience.
NLnet Labs is embarking on a new adventure. In the coming months we will be developing an RPKI toolset aimed at making BGP routing more secure. It will consist of three parts: a Certificate Authority package, a Publication Server and Relying Party software. We’ve chosen to do the development in a modern systems programming language, Rust.
With the latest release of Amp (text editor), syntax highlighting has been overhauled. This post dives into the performance-related redesign that has landed with v0.5.
Ever had some code base that you regularly use to start a new project? Until now, you've probably lost some time refactoring everything to fill out the right project name, title etc. A few tools already exist but either you're lost in feature bloating hell or you're fighting to configure everything in most cases. The following tool is an early-developement, rust-based, template tool made to be more versatile that its closest python counter-part, cookiecutter.
Don’t waste time waiting for your bundler to do its thing. Use Pax while you’re developing, and iterate to your heart’s content. Use your super-cool, magical, slow-as-molasses bundler for releases, when you don’t care how long it takes to run.
glitchcat is a cat-like program with glitch animation.
So, you collect metrics. So do we. Yep, we also collect them. We can assure you that they are not useless at all, business guys do need…
CLion 2018.2 was released just recently. Along with the release, the Rust plugin has had an update! In this blog post, we’d like to highlight some of the main improvements in the plugin.
Gutenberg 0.4.0 is out with custom taxonomies, image processing, improved shortcodes and more.
mazon S3 is a storage solution used by pretty much everyone these days. Due to this there are naturally a bunch of tools for doing almost everything you can think of with S3. This post is about a tool I wrote to retrieve metadata about S3 buckets. If you don't care for the post, feel free to skip straight to the repo.
This past fall, three former GnuPG developers began working on a new OpenPGP implementation in Rust called Sequoia. As it’s starting to shape up and become useful, I feel now is a good time to announce the project to the larger Rust community, and hopefully get some feedback before our first release.
In this post, we’re going to build a sequence-based recommender system in Rust: a system that accepts a person’s reading history as input, and outputs recommendations on what to read next.
The piano arrangement album 帰るべき城 by Altneuland was published in 2005. I discovered it in 2008 (probably on YouTube), downloaded the best copy I could find, and filed it away in the TODO list. Recent advances in international parcel forwarding technology let me buy a used copy last year, but when it arrived none of my CD drives could read track #3. This sort of thing is common when buying used CDs, especially if they need to transit a USPS international shipping center. I shelved it and kept on the lookout for another copy, which I located last month. It arrived on Friday, I immediately tried to rip it, and hit the exact same error. This didn’t seem to be an issue of wear or damage…
Lots of people asked me to write another piece about the internals of well-known Unix commands. Well, actually, nobody asked me, but it makes for a good intro. I'm sure you’ve read the previous parts about yes and ls — they are awesome.
Anyway, today we talk about cat, which is used to concatenate files - or, more commonly, abused to print a file's contents to the screen.
We are excited to announce the next version of vagga, the containerization tool that makes setting up development environments easy and fun. This release brings local overrides for vagga commands, support of ubuntu bionic, better CI support, space reuse between multiple projects, and more.
cargo-generate is a developer tool to help you get up and running quickly with a new Rust project by leveraging a pre-existing git repository as a template.
We recently had the opportunity to make a Visual Studio Code extension that needed to communicate with an embedded device. This post explains the architecture we chose to achieve that and the decisions that led to it.
Almost every project I’ve worked on has grown a shell script named “build.sh”, and not much later a “test.sh” and “run.sh”. At this point, you have to make a decision as a developer whether your goal is to accidentally reinvent make or if your codebase’s needs are better met by an executable to manage your workflow.
Nannou is an open source creative coding framework built in Rust for artists, hackers, designers and devs to express themselves with simple, fast, reliable, portable code.
As operator I found that existing tooling fails with large number of unusual devices. If you have server with >100 virtual machines, and each virtual machines have one or more additional disk and a few network interfaces, it’s hard to find which VM cause stress on the server. Normal atop simply couldn’t cope with hundreds of tap/tun interfaces, multitude of block devices, etc. When I found that I have no proper tooling (as an operator) I wrote a shabby tools to do my job: they are calling blktop and ifstop. Both were written dirty and fast, and they does not follow the usual Python conventions (but they work as standalone binaries). They do their job, and normally I would say that’s enough. But those utilities are simple and have no big complications under the hood, so they are perfect candidates for rewriting in Rust for the sake of writing in Rust (I’m learning it!). I decide to combine them into a single utility: devtop, which I want to develop properly, according to Rust conventions, and through all aspects of packaging.
It's like SSH, but more secure, and with cool modern features. It is not an implementation of SSH, it is a new, modern protocol.
Finda lets you search and instantly switch between tabs in browsers like Firefox and Chrome and editors like Sublime Text and Visual Studio Code. It does this by communicating with these programs over local websocket connections. This article describes the design tradeoffs of this system, focusing on Rust implementation details that may be helpful to others building similar systems.
I am currently working on Wayk Now written in C for the most part. We recently started writing new code for it in Rust which I’m super excited about.
As we needed a way to integrate this code in our CMake build1, we wrote a set of modules for CMake
I am quite notorious for exploiting Gitlab’s CI. Ever since I started playing with it at the start of 2016, I tended to make things worse.
The C2Rust project is being developed by Galois and Immunant. This tool is able to translate most C modules into semantically equivalent Rust code. These modules are intended to be compiled in isolation in order to produce compatible object files. We are developing several tools that help transform the initial Rust sources into idiomatic Rust.
In this post, I will walk you through configuring an EMACS Rust IDE to get syntax highlighting and code completion for the Rust programming language.
Last week I tweeted "What do you think are the most interesting/exciting projects using Rust? (No self-promotion :-) )". The response was awesome! Jonathan Turner suggested I write up the responses as a blog post, and here we are.
Rust Qt Binding Generator lets you combine Rust code with a Qt1 graphical application. A previous blog shows how to make a simple clock. It’s a good idea to read that post before reading this more advanced post, because in this post we are getting serious.
This blog post shows how to write a to-do application. The data model is a list of to-do items. The source code for this example is available in the folder examples/todos in the Rust Qt Binding Generator repository.
Ahead of this year’s World Password Day, 1Password – maker of password management software – announced a password cracking challenge. The company ostensibly wanted to find out how hard it would be to crack a three-word passphrase master password on one of their vaults, assuming that the attacker had the derived hash of the passphrase.
Back in Februari 2013 then coworker Romain S. showed me the new trend of programming editors that do continuous compilation while you type, showing you immediate feedback on your code. In parallel I also worked on 3D modeling for my 3D printer using the OpenSCAD program. OpenSCAD works by writing code in its custom language and then have it rendered. I had this idea of combining these two approaches to make an electronics footprint generator. And so the development of the original madparts program started. In August 2016 I had been playing with the then pretty new rust programming language and decided a rewrite in it and simplifying the program even further would be fun to do.
Small tutorial of using Smithay's Client Toolkit by example, building a small image viewer as a Wayland client.
dutree is a command line tool to analyze disk usage. Features coloured output, according to the LS_COLORS environment variable, display the file system tree, ability to aggregate small files, ability to exclude files or directories, ability to compare different directories fast, and written in Rust. This tool is a mix between tree and durep.
For a fun project, I’ve been tinkering with xi-win, an experimental Windows front-end for xi-editor, written in Rust. I’m basically optimizing for performance, so making a number of somewhat unusual decisions. Among other things, I’m writing the UI myself, rather than using an existing toolkit or framework.
sudo is used by engineers daily to run commands as privileged users. But on some sensitive systems, you really want to ensure that no individual can act entirely autonomously. At Square, this includes applications that manage our internal access-control systems, store accounting ledgers, or even move around real money. This plugin allows us to ensure that no user can act entirely on their own authority within these systems.
I’ve been thinking about information theory, entropy, and passphrases for a couple of months now. I’ve been particularly interested in using random passphrases as passwords. An example of one of these passphrases would be “stamina turret backlands ruby”. The words have to be as purely random as possible – using your four dogs’ names is not nearly as strong as a password, as an attacker would likely guess that relatively early.
Whilst sort -u is super easy to use, the requirement of sorting your data cannot be overlooked as it means your entire file needs to be buffered into memory at once. This is not always possible, or even if possible it's not always desired. On the other hand uniq is great, but in order to sort your data in advance you still have the same issue. It was due to this that I wrote runiq, which is essentially an optimized and more flexible implementation of both of the aforemention tools for the purposes of duplicate filtering. It performs much faster, with much less memory, and does not require sorted input (although it can optimize this case).
TL;DR seiri is a lightweight music manager written (mostly) in Rust that helps keep huge music libraries organized. Try it today at chyyran.moe/seiri.
This is a web-based Kanban board application, built with Elm and Rust. The only different from this application and hundred thousands of web-based application out there is: We don't need Electron!. Instead, it use native WebView (WebKit for Linux/macOS, and MSHTML on Windows),
In 2012, Mozilla started the Servo project, a community effort to create new open source browser components that can take advantage of multicore hardware to improve speed, stability and responsiveness. Servo is comparable to WebKit, an open source browser engine that is the foundation of Apple’s Safari browser. (Other browser engines include Blink, EdgeHTML, and Gecko.) The difference is that Servo is written in the Rust programming language, bringing lightning-fast performance and memory safety to browser internals. Rust code can drastically cut down the number of critical bugs that affect browser engines. Servo uses a modular architecture and currently runs on Linux, macOS, Windows, and Android operating systems.
bat supports syntax highlighting for a large number of programming and markup languages. bat also communicates with git to show modifications with respect to the index (see left side bar).
Way back in the dim mists of history (back in university) I wrote myself a custom RSS reader in Java and called it JARVIS1. You see, I read a lot of webcomics. Like, a lot. Some webcomics provide RSS feeds, but some don’t, and as my collection grew it started to become a hassle to use Firefox’s live bookmarks to manage it all. Ultimately, I wrote up a quick Swing GUI to use as a single interface for keeping up with blogs and tracking which comics had published updates since the last time I’d checked2.
Hammond has been in development since Sep 2017. It makes use of the Rust bindings to GNOME libraries as well as diesel, serde, reqwest, and a number other crates to build a beautiful, native application that feels right at home on the GNOME desktop.
fastmod is a fast partial replacement for codemod. Like codemod, it is a tool to assist you with large-scale codebase refactors, and it supports most of codemod's options. fastmod's major philosophical difference from codemod is that it is focused on improving the use case "I want to use interactive mode to make sure my regex is correct, and then I want to apply the regex everywhere". For this use case, it offers much better performance than codemod
I’m pleased to announce the new release of Pijul, version 0.10. Pijul is an easy to use, distributed and fast version control system. This release has been a long time coming, but brings in an important number of new features and stability enhancements.
cargo src is a new tool for exploring your Rust code. It is a cargo plugin which runs locally and lets you navigate your project in a web browser. It has syntax highlighting, jump to definition, type on hover, semantic search, find uses, find impls, and more.
Dust is meant to give you an instant overview of which directories are using disk space without requiring sort or head. Dust will print a maximum of 1 'Did not have permissions message'. Dust will list the 15 biggest sub directories or files and will smartly recurse down the tree to find the larger ones. There is no need for a '-d' flag or a '-h' flag. The largest sub directory will have its size shown in red
Angle-grinder allows you to parse, aggregate, sum, average, percentile, and sort your data. You can see it, live-updating, in your terminal. Angle grinder is designed for when, for whatever reason, you don't have your data in graphite/honeycomb/kibana/sumologic/splunk/etc. but still want to be able to do sophisticated analytics.
Angle grinder can process about a million rows per second, so it's usable for fairly meaty aggregation. The results will live update in your terminal as data is processed. Angle grinder is a bare bones functional programming language coupled with a pretty terminal UI.
A few months ago, Bünz, Bootle, Boneh, Poelstra, Wuille, and Maxwell published Bulletproofs, which dramatically improves proof performance both in terms of proof size and verification time. In addition, it allows proving a much wider class of statements than just range proofs.
At Chain, we (Henry de Valence, Cathie Yun and Oleg Andreev) have been working on a pure-Rust Bulletproofs implementation, whose initial version we are publishing today, together with a set of notes.
A software library for high performance and hardware accelerated simulation of Quantum Computers and Algorithms. Written with Rust and OpenCL.
This new version comes with great performance improvements. We're talking about 3x faster on macos, 2x faster on linux and 3x faster on windows (the benchmarks are at the end of the post).
Sequoia is a modular OpenPGP implementation in Rust.
pfr is a command-line tool for helping me to manage my personal finances. At the end of each month, I like to sit down and “allocate” funds to cover my expenses for the following month. However, not all of my expenses occur monthly. For example, things like rent are paid weekly, and I think of money spent on food in terms of weeks, so I have to multiply those expenses by 4 (ish). I also have to figure out how much money to put in each account, e.g on my EFTPOS (direct-debit) card or for automatic payments. It’s not hard, but then again, that’s why we write programs.
Click is an open-source tool that lets you quickly and easily run commands against Kubernetes resources, without copy/pasting all the time, and that easily integrates into your existing command line workflows. This blog introduces its its usage and merits.
I wish I could have the features of both tools, but for now I will use gutenberg over cobalt. A tiny migration and feature comparison story.
(or how rust helped me to match the speed of GNU join written by Mike Haertel)
Xray - An experimental next-generation Electron-based text editor. This update covers a revised roadmap, fast file finding, and thoughts on key bindings.
In case you have tried to make a rust program with GUI that would work both on Linux and Windows, you probably may have faced some issues in some point (especially if you are a newcomer). Compiling in Linux is quite easy, but in Windows life gets a bit tricky.
I've pulled together some info from here and there so that no-one else had to fight their way through fire and flames.
A tool for serving and managing presentation slides.
When I built Finda, I wanted it to be fast — specifically, to respond to all user input within 16 milliseconds.
Given this goal, you might be surprised to learn that Finda is built with Electron, a framework that’s often decried for being the opposite of fast.
In my series of useless Unix tools rewritten in Rust, today I'm going to be covering one of my all-time favorites: ls.
In recent years, it has become increasingly important to develop software that minimizes security vulnerabilities. Memory management bugs are a common cause of these vulnerabilities. To that end, the Mozilla community has spent the last several years building the Rust language and ecosystem which focuses primarily on eliminating those bugs. And Rust is available in Fedora today, along with a few applications in Fedora 27 and higher, as seen below.
A bit later than anticipated, this is now part two of the blog post series about writing GStreamer elements in Rust. Part one can be found here, and I’ll assume that everything written there is known already. In this part, a raw audio sine wave source element is going to be written. It will be similar to the one Mathieu was writing in his blog post about writing such a GStreamer element in Python. Various details will be different though, but more about that later.
The team at Paris-based Snips has created a voice assistant that can be embedded in a single device or used in a home network to control lights, thermostat, music, and more. You can build a home hub on a Raspberry Pi and ask it for a weather report, to play your favorite song, or to brew up a double espresso. Manufacturers like Keecker are adding Snips’ technology to products like multimedia home robots. And Snips works closely with leaders across the value chain, like NVIDIA, EBV, and Analog Devices, in order to voice-enable an increasingly wider range of device types, from speakers to home automation systems to cars.
spectra is a crate I’ve been maintaining for a few months / years now. It’s a crate that I mainly use for demoscene productions (I released two with it, Céleri Rémoulade and Outline Invitation) but I also use it to play around and experiment new rendering, animation and video game techniques.
Fanta is a web framework that aims for developers to be productive and consistent across projects and teams. Its goals are to be: Opinionated, Fast, and Intuitive. Based heavily off of the work here: https://github.com/tokio-rs/tokio-minihttp
comm is a peer-to-peer instant messaging protocol designed to be resilient to censorship. comm-gtk is a GUI client built on the comm library. To try it out, start the app. In the configuration window, enter a secret phrase, a bootstrap node (IP:port pair), and a local port to listen on (e.g. 6669). For a bootstrap node, try 184.108.40.206:6667 (or any other node's IP address if you know one). Click connect to join the network.
A fast command line client for tldr: A collection of simplified and community-driven man pages.
I had an itch: I was pretty-printing the BERT-encoded terms that we use in a production system at work and it was very slow. The Erlang shell took more than two minutes to dump the largest file. (It took about 0.1 second to read and parse the file; the rest was spent in io:format.) I decided to scratch that itch: I wrote ppbert, a command-line utility that reads BERT-encoded values and pretty-prints them. I’ve worked sporadically on ppbert for almost a year now, I use it daily at work, I’m happy with it, and I want to write about some of the things I learned during that journey.
In an effort to experiment with dsp, I wrote a guitar/bass effects processor this past weekend. The end result works very well (to my pleasant surprise). It doesn’t have 90% of the functionalities of any of rakarrack, guitar rig, garage band but overall it was a fun weekend hack.
DataFusion is an open-source Big Data platform implemented in the Rust programming language with a similar programming style to Apache Spark.
retrobasic is a BASIC interpreter written in Rust. It is based on the original BASIC '64 implementation, and modified as needed to make it more compatible with some of the classic BASIC games.
As a library writer, it feels a bit strange, but refreshing, to write a program that actually has a main() function. My experience with Rust so far has been threefold:
An obvious idea is to use Rust for task automation. Originally, I have proposed creating a special Cargo subcommand to execute build tasks, implemented as Rust programs, in this thread. However, since then I realized that there are built-in tools in Cargo which allow one to get a pretty ergonomic solution. Namely, the combination of workspaces, aliases and ability to define binaries seems to do the trick.
Cat for markdown: Show markdown documents in TTYs