I have heard many good things about Rust for several years now. A couple of months ago, I finally decided to start learning Rust. I skimmed through the Book and did the exercises from rustlings. While they helped me get started, I learn best by doing some projects. So I decided to replace the crawler that I used for my Ghost blog, which had been written in bash with wget, with something written in Rust.
And I was pleasantly surprised. I am by no means very knowledgeable in Rust, I still have to look up most of the operations on the Option and Result types, I have to DuckDuckGo how to make HTTP requests, read and write files and so on, but I was still able to write a minimal crawler in about 2-3 hours and then in about 10 hours of total work I had something that was both faster and had fewer bugs than the wget script.
So let's start writing a simple crawler that downloads all the HTML pages from a blog.
Today Friedel Ziegelmayer (Protocol Labs), Ryan Levick (Microsoft), and myself would like to introduce a new set of HTTP libraries to make writing encrypted, async http/1.1 servers and clients easy and quick:
* async-h1 – A streaming HTTP/1.1 client and server protocol implementation.
* http-types – Reusable http types extracted from the HTTP server and client frameworks: Tide and Surf.
* async-native-tls – A streaming TLS client and server implementation.
With these libraries writing a streaming, encrypted HTTP client takes about 15 lines of code.
Back in 2014 I was fetching frontpages of the top million websites to scan them for a particular vulnerability. Not only have I found 99,9% websites to be vulnerable to a trivial attack, I’ve also found that curl command was randomly crashing with a segmentation fault, indicating a likely vulnerability in libcurl — the HTTP client library that the whole world seems to depend on.
By that time I was already disillusioned in the security of software written in C and the willingness of maintainers to fix it, so I never followed up on the bug. However, this year I decided to repeat the test with software written in a language that’s less broken by design: Rust.
Here’s how 7 different HTTP clients fared.
reqwest is a higher-level HTTP client for Rust. Let me introduce you the v0.10 release that adds async/await support!
reqwest alpha.await reqwest is a higher-level HTTP client for Rust. I’m delighted to announce the first alpha release that brings async/await support!
Today we're happy to announce Surf, an asynchronous cross-platform streaming HTTP client for Rust. This project was a collaboration between Kat Marchán (Entropic / Microsoft), Stjepan Glavina (Ferrous Systems), and myself (Yoshua Wuyts). Surf is a friendly HTTP client built for casual Rustaceans and veterans alike.
One of my more recent projects that I have been putting a lot of effort into is a Rust HTTP client called cHTTP, which I introduced on this blog over 18 months ago. Here I want to share an update on the direction of the project, and also give some detail on what months of late evenings and weekends produced in version 0.5 just published today.
View all tags