edg-l/edgarluque.com
1
0
Fork 0
mirror of https://github.com/edg-l/edgarluque.com.git synced 2024-11-09 17:48:22 +00:00
Code Issues Packages Projects Releases Wiki Activity

image

Browse source
This commit is contained in:
Edgar 2022-01-06 10:41:59 +01:00
parent 73b304f902
commit 2eeb446d53
No known key found for this signature in database
GPG key ID: 8731E6C0166EAA85
1 changed files with 10 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

12
content/blog/zstd-streaming-in-rust.md
View file

@ -30,7 +30,7 @@ We will use the following libraries to achieve this:
## Fetching the data ## Fetching the data
```rust ```rust
let resp = ureq::get("https://ddnet.tw/stats/master/2022-01-05.tar.zstd").call()?; let resp = ureq::get("https://ddnet.tw/stats/master/2022-01-04.tar.zstd").call()?;
// Read the content length from the header. // Read the content length from the header.
let len: usize = resp.header("Content-Length").unwrap().parse()?; let len: usize = resp.header("Content-Length").unwrap().parse()?;
@ -86,4 +86,12 @@ let data: ServerList = simd_json::from_reader(entry).expect("parse json");
This way, we are parsing each file efficiently while using almost no RAM thanks to the streaming nature of these operations. This way, we are parsing each file efficiently while using almost no RAM thanks to the streaming nature of these operations.
This all fits really well thanks to the design of [Read](https://doc.rust-lang.org/std/io/trait.Read.html) and [Write](https://doc.rust-lang.org/std/io/trait.Write.html). This all fits really well thanks to the design of [Read](https://doc.rust-lang.org/std/io/trait.Read.html) and [Write](https://doc.rust-lang.org/std/io/trait.Write.html).
## The tool
Here is the source code of the tool: <https://github.com/edg-l/teemasterparser>
And an image of the result:
<img src="https://github.com/edg-l/teemasterparser/raw/master/example.svg" width="100%">