Tag Archives: Rust

Sqlite3 – How Slow Is Write?

Sqlite3 is lightweight relational database, mainly focused on smaller local systems. Being used in Android it’s now probably most spread relational database in world with billions of instances running. Lite in the name means that it is not client-server architecture and it’s intended for lower data volumes – ideal usage profile is read mostly, with occasional writes. Sqlite3 is often used as an embedded data store in various applications (Firefox and Chrome are most prominent ones). Recently I’ve been playing a bit with sqlite3 interface in Rust and had run couple of simple tests especially focused on writes. So how does sqlite3 performs and how it compares with other more typical client-server RDBMS like PostgreSQL?  It’s not any serious benchmark, just couple of toy tests to highlight few things. Continue reading Sqlite3 – How Slow Is Write?

Future Never Sleeps

Recently I’ve been reading this book:  “Network Programming with Rust” by Abhishek Chanda. I found this book bit problematic. It’s just collection of many unrelated examples (often taken from crates documentation), with just little of background and concepts explanation and in some parts this book is just wrong, in other parts it’s using too much simplifications, so the result does not make much sense or worst it introduces some dangerous ideas. One of these  places is part about futures and streams – let’s look at one example: Continue reading Future Never Sleeps

Fearless Upgrades with Typed Languages

One of many advantages of statically typed programing languages like Rust or Java is their resilience to changes in dependent libraries, usually caused by new library versions with modified interface – e.g. the major version changes. In statically typed language we usually see problems in compile time and it should be relatively easy to fix, but in dynamic languages such as Python or Javascript upgrade is more challenging and problems  demonstrate themselves during tests (in better case, when we have good test coverage) or in production in worst case. I had recently came through this process for couple of  relatively small projects in Rust. Couple of dependencies (hyper and tokio) had been upgraded to new versions with significant changes to their interface. With update compilation broke with tens of errors, but gradually I was able to fix all of them (in one case it required to improve error handling with custom error type, plus using additional new library, as typed headers were removed form hyper) and after code compiled and run through basic tests I was pretty sure that I’m fine with new libraries. In similar situation in python I would need much more work to be sure that code works after such major upgrade of dependencies. In practice it enables easier maintenance of code and less effort  to keep it up to date with latest libraries. For library authors it provides more freedom and they can introduce breaking changes more often (with great cargo build tool in Rust library users can pin themselves to older version, if they do not want to upgrade).

From Ignorance to Enlightenment – Playing with Tokio

I have been playing with tokio already in couple of small projects (ptunnel-rust and indirectly (via hyper) in audioserve), but I cannot say that I’m proficient.  Also tokio is very much moving target –  what I used couple month ago is already bit outdated now(old version is tokio_core crate – where default executor was on current thread, now it’s work stealing thread pool). So I decided to refresh and deepen my knowledge and created a toy project –  stupid jokes server –  it’s a TCP sever, which sends a random  joke to client after it connects and then closes connection. Jokes are stored in text file, separated by dashed lines.  My main interest was to test how to use local file system I/Os, which are blocking by nature, with tokio asynchronous approach (so I initially skipped easiest and probably most efficient implementation, where all jokes would be cached in memory).  Usually in a real project you’ll have some blocking code, so I need to know how to handle it. This article is history of my attempts (and failures) recorded in a hope that it might help others in learning tokio (and also writing it down helped me to absorb gained knowledge). Continue reading From Ignorance to Enlightenment – Playing with Tokio

How much better is the thread pool?

Is thread pool worth to consider for my project?   I was  looking for some opinions around the net and as usual they  differ and most common wisdom is it matters. Generally it’s “known” that creating and tearing down thread is “significant” overhead, so if you have a lot of small tasks thread pool is much better solution then spawning new thread for each task.  But what is significant overhead?  According to what I read time to create thread on Linux should be about 10μs (which does not look as too much to me) and app. 2MB of memory allocated for stack (configurable).   I was considering thread pool in context of audioserve project, where I started with simplest possible solution (e.g. spawning individual threads ) and was wondering how much I’m loosing by not using thread pool. So I implemented simple thread pool (as learning exercise – long term audioserve solution should use tokio-threadpool)  and add it to audioserve.  In the remainder of this short article I’d like to share my findings and  roughly quantify benefits of thread pool for such small project. Continue reading How much better is the thread pool?

Asynchronous Again – Rewriting ptunnel in Rust

Asynchronous programing model is quite popular for I/0 intensive tasks – it enables you effective use of resources, while maintaining agility of and assuring scalability of the application. I myself used asynchronous programming many times –   in JavaScript (where it’s omnipresent) , Python ( mainly  in asyncio recently, but also bit in twisted, which was one of first network asynchronous libraries I met) and also in OCAML with  lwt or Core Async. The concept is always similar for all implementations –  I/O operations are returning handles to future results – they are called either  Futures, Promises, or Deferred  – and they are returned immediately.  These futures can have functions attached to them, which are executed later, when I/O result becomes available.  Asynchronous  programming is very much about functions, it requires first class functions  and anonymous functions are very useful here, that’s why asynchronous model flourishes in functional languages.  Apart of I/O deferred processing usually there are other utilities for later execution – like timeouts, pausing execution for some time (sleep), tasks synchronization (events, locks). Futures are executed in an “event loop”,   a loop that monitors various events from OS (availability of data from I/O), timers, etc. to execute futures (meaning functions attached to them), when appropriate. It’s also very common to chain futures, executing second one with result of first one , when first one is resolved and result is available and the third one with results from the second one and so on. Apart of this basic scheme languages may provide some syntactic sugar around asynchronous model like await and async keywords in Python or C#, which makes it easier to write the code.

Recently, as I’m progressing in learning of Rust,  I wondered how asynchronous programing is done in Rust. I decided to remake my old project ptunnel (written in Python) into Rust – ptunnel is a program that tunnels arbitrary connection/protocol through HTTPS proxy, so it can be used to connect IMAP, SMTP or SSH through proxy. In the rest of this article I”l share my experiences from this project. Continue reading Asynchronous Again – Rewriting ptunnel in Rust

Secret Sharing Is Caring Too

In todays digital world passwords and other types of secrets are omnipresent and they secure access to various assets dear to our hearts, some of those can have tremendous tangible or moral value. For such assets it’s worth to select really good and strong password, which basically means long and hard to remember. How to ensure ourselves in case of memory failure? We can write it down and lock in secure place, share with trusted person etc., but still there is one point of of failure – secure place can be robbed, that person can betray us. Can cryptography  provide us with better options?  Yes it can with help of method called Secret sharing – we can split secret into n parts – called shared secrets – and distribute them to different places/people. Later we (or someone else) need to collect k (k > 0 and k <= n) shared secret to recover original secret. k is called threshold and it is defined when generating shared secrets – so we for instance generate n=5 shared secrets, but only k=3 will be needed to recover original secret.

I believe you can easily imagine  many other real life scenarios where secret sharing can be useful and for sure it’s used in many applications and systems today. Cryptography provides several algorithms for secure (by design) secret sharing.  Most common is Shamir’s Secret Sharing based on linear algebra approach. There are many tools and libraries for Shamir’s scheme (and further advancements of original algorithm),  you can for instance try ssss, which provides command line tool that you can easily install into your Linux and also there is an online demo. Another family of secret sharing schemes is based on Chinese Reminer Theorem, where especially Asmuth-Bloom scheme is interesting.  I have not seen many implementation for Asmuth-Bloom secret sharing so I created one in Rust. Continue reading Secret Sharing Is Caring Too

In RUST We Trust

Having been programing recently mostly in dynamic untyped languages (aka Python and JavaScript) I though that it would be nice to try something else, bit different –  meaning compiled and statically typed. Last adventures in this area were with OCAML, which I used for few learning projects couple years ago( like this one).  OCAML is very nice language indeed, and learning exercise was very valuable for me (especially getting more accustomed to functional programming style),  but apart  of that learning experience I did not follow it  further (mainly due limited ecosystem of OCAML).

Looking recently to languages and technology survey on Stackoverflow  where Rust is leading the list of most “loved” languages (meaning developers who used the language like it and want to use it for their next projects) with head start on   second one (SmallTalk) .   This caught my attention and looking quickly at Rust site I decided to give it a try.  Below are my first experiences learning this language. Continue reading In RUST We Trust