Building JSON-RPC APIs in Rust
Tomasz Drwięga
@tomusdrw
Parity Technologies
25 July 2018, Wrocław Rust Meetup
JSON-RPC
{
"jsonrpc": "2.0",
"id": 1,
"method": "getTransactions",
"params": [{ "since": 1532502205406 }]
}- Stateless
- Transport agnostic
- Remote Procedure Call protocol
Request
JSON-RPC
{
"jsonrpc": "2.0",
"id": 1,
"result": []
}Response
{
"jsonrpc": "2.0",
"id": 1,
"error": {
"code": 32000,
"message": "Something went wrong",
"data": ["Validation error 1"]
}
}Why JSON-RPC?
- Simple
- Transport agnostic
- Support in various languages
- Standardized errors
- Standard in crypto-space
(Bitcoin, Ethereum)
What we do at Parity?
- Parity Ethereum - Ethereum Client
- Parity Bitcoin - Bitcoin Client
- WASM (wasmi)
- Whisper
- IPFS / libp2p
- Polkadot - The Internet of Blockchains
- Parity Substrate
- Many RPCs exposed :)
Everything built with Rust (+ some JS).
crates.io/jsonrpc-core
- Rust library to easily build JSON-RPCs
- Core + Transports
- Async support (futures)
- Serde-based
- Couple of useful extensions
Dude, show the code!
extern crate jsonrpc_core as rpc;
#[test]
fn should_send_hello() {
// given
let mut io = rpc::IoHandler::new();
io.add_method("say_hello", |_params: rpc::Params| {
Ok(rpc::Value::String("hello".to_string()))
});
// when
let response = io.handle_request_sync(r#"
{
"jsonrpc": "2.0",
"id": 1,
"method":"say_hello"
}
"#);
// then
assert_eq!(response, Some(
r#"{"jsonrpc":"2.0","result":"hello","id":1}"#.into()
));
}Transports
- HTTP (hyper-based)
- MiniHTTP (tokio-minihttp)
- WebSockets (rewrite upcoming)
- IPC (Windows & *nix)
- TCP
HTTP server
extern crate jsonrpc_core as rpc;
extern crate jsonrpc_http_server as http;
fn main() {
let mut io = rpc::IoHandler::new();
io.add_method("say_hello", |_params: rpc::Params| {
Ok(rpc::Value::String("hello".to_string()))
});
let address = "127.0.0.1:3030".parse()
.expect("Valid address given");
let server = http::ServerBuilder::new(io)
.threads(3)
.start_http(&address)
.expect("Server should start");
server.wait()
}Microbenchmarks
# JSON-RPC minihttp (3 threads)
$ cargo run
$ wrk -t 4 -c 4 http://localhost:3030
Running 10s test @ http://localhost:3030
4 threads and 4 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 183.91us 91.04us 3.45ms 90.69%
Req/Sec 5.50k 1.64k 7.73k 53.96%
221195 requests in 10.10s, 44.93MB read
Non-2xx or 3xx responses: 221195
Requests/sec: 21900.60
Transfer/sec: 4.45MB
# Node
$ ./node.js
$ wrk -t 4 -c 4 http://localhost:3000
Running 10s test @ http://localhost:3000
4 threads and 4 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 150.06us 201.16us 10.71ms 98.54%
Req/Sec 7.05k 0.95k 8.53k 88.12%
283229 requests in 10.10s, 37.55MB read
Requests/sec: 28042.79
Transfer/sec: 3.72MBMicrobenchmarks
# JSON-RPC minihttp (3 threads)
$ cargo run --release
$ wrk -t 4 -c 4 http://localhost:3030
Running 10s test @ http://localhost:3030
4 threads and 4 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 53.57us 54.91us 3.12ms 99.46%
Req/Sec 19.05k 2.35k 23.99k 67.57%
765634 requests in 10.10s, 155.53MB read
Non-2xx or 3xx responses: 765634
Requests/sec: 75812.61
Transfer/sec: 15.40MB
# JSON-RPC minihttp (3 threads) with payload
$ cargo run --release
$ wrk -t 4 -c 4 --script ./script.lua http://localhost:3030
Running 10s test @ http://localhost:3030
4 threads and 4 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 74.00us 42.98us 2.85ms 98.49%
Req/Sec 13.52k 1.50k 15.59k 52.72%
543735 requests in 10.10s, 87.63MB read
Requests/sec: 53835.55
Transfer/sec: 8.68MBjsonrpc-macros
extern crate jsonrpc_core as rpc;
#[macro_use]
extern crate jsonrpc_macros;
use rpc::Result;
build_rpc_trait! {
pub trait Rpc {
/// Adds two numbers and returns a result
#[rpc(name = "add")]
fn add(&self, u64, u64) -> Result<u64>;
}
}
pub struct RpcImpl;
impl Rpc for RpcImpl {
fn add(&self, a: u64, b: u64) -> Result<u64> {
Ok(a + b)
}
}
fn main() {
let mut io = rpc::IoHandler::new();
io.extend_with(RpcImpl.to_delegate());
}async
extern crate jsonrpc_core as rpc;
#[macro_use]
extern crate jsonrpc_macros;
use rpc::{BoxFuture, futures::{self, Future}};
build_rpc_trait! {
pub trait Rpc {
/// Adds two numbers and returns a result
#[rpc(name = "add")]
fn add(&self, u64, u64) -> BoxFuture<u64>;
}
}
pub struct RpcImpl;
impl Rpc for RpcImpl {
fn add(&self, a: u64, b: u64) -> BoxFuture<u64> {
let (tx, rx) = futures::oneshot();
::std::thread::spawn(move || {
tx.send(a + b)
});
Box::new(
rx.map_err(|_| unreachable!())
)
}
}Extensions
- Metadata
(extract transport-specific data) - Middlewares (intercept calls)
- Test Utils (no Strings in tests)
- Publish-Subscribe
(subscribe to notifications)
PubSub
extern crate jsonrpc_core as rpc;
extern crate jsonrpc_pubsub as pubsub;
#[macro_use]
extern crate jsonrpc_macros as macros;
use rpc::Result;
build_rpc_trait! {
pub trait Rpc {
type Metadata;
#[pubsub(name = "hello")] {
/// Hello subscription
#[rpc(name = "hello_subscribe", alias = ["hello_sub", ])]
fn subscribe(&self, Self::Metadata, macros::pubsub::Subscriber<String>, u64);
/// Unsubscribe from hello subscription.
#[rpc(name = "hello_unsubscribe")]
fn unsubscribe(&self, pubsub::SubscriptionId) -> Result<bool>;
}
}
}
#[derive(Default)]
struct RpcImpl {
uid: atomic::AtomicUsize,
active: Arc<RwLock<HashMap<pubsub::SubscriptionId, macros::pubsub::Sink<String>>>>,
}
impl Rpc for RpcImpl {
fn subscribe(&self, _meta: Self::Metadata, subscriber: _, _param: u64) {
let id = self.uid.fetch_add(1, atomic::Ordering::SeqCst);
let sub_id = pubsub::SubscriptionId::Number(id as u64);
let sink = subscriber.assign_id(sub_id.clone()).unwrap();
self.active.write().unwrap().insert(sub_id, sink);
}
Thank you
Tomasz Drwięga
@tomusdrw
What is Blockchain?
- Distributed data structure - to organize "transactions"/events
- Consensus Algorithm - to decide who is allowed to modify that structure
- Some crypto - to make it secure/tamperproof
- Incentives - to make it run by itself
List of changes
List of changes
List of changes
Metadata
/ Previous state
Metadata
/ Previous state
Metadata
/ Previous state
Genesis State
Blockchain
Immutable data structure containing all the changes that were applied in any point in time
How does it work?
(Boot) Node 1
Node 2
New Node
Hey! Could you give me all your peers?
How does it work?
(Boot) Node 1
Node 2
New Node
Hey! Send me all them blocks, will ya?
Block 5
Block 4
Block 0
How does it work?
(Boot) Node 1
Node 2
New Node
Hey! I've got a transaction to include in block.
Block 5
Block 5
Block 5
transfer(N, B, 5)
sig(N)
How does it work?
(Boot) Node 1
Node 2
New Node
Block 5
Block 5
Block 5
transfer(N, B, 5)
sig(N)
transfer(N, B, 5)
sig(N)
Cool, I'm mining and will include the tx for a small fee.
How does it work?
(Boot) Node 1
Node 2
New Node
Block 6
Block 5
Block 5
transfer(N, B, 5)
sig(N)
transfer(N, B, 5)
sig(N)
Block 6
Managed to mine new block, here it is guys!
How does it work?
(Boot) Node 1
Node 2
New Node
Block 6
Block 6
Block 6
List of changes
List of changes
List of changes
Metadata
/ Previous state
Metadata
/ Previous state
Metadata
/ Previous state
Genesis State
Blockchain
Immutable data structure containing all the changes that were applied in any point in time
Blockchain
Hashes - prevent tampering (e.g. KECCAK256)
Signatures - authorize the actions (e.g. ECDSA)
Parent = hash(B0) Timestamp = 150..000 Number = 1
Hash = hash(B1)
transfer(A, B, 5)
sig(A)
transfer(C, B, 1)
sig(C)
Parent = hash(B2) Timestamp = 150..000 Number = 2
Hash = hash(B1)
transfer(B, A, 5)
sig(B)
Consensus Algorithm
Who is allowed to create new blocks?
sig(Authority1)
hash(B0)
hash(B1)
sig(Authority2)
hash(B2)
sig(Authority1)
Proof of Authority
We only accept blocks signed by a hardcoded list of authorities.
Blocks need to be signed in turns at most 1 block per 3 seconds.
Consensus Algorithm
Who is allowed to create new blocks?
Difficulty=2 Sol.=0b001.. SolvedBy=A
hash(B0)
hash(B1)
hash(B2)
Proof of Work
We only accept blocks with a solution to a puzzle.
The difficulty of the puzzle can be adjusted to have a stable rate of new blocks.
Difficulty=4 Sol.=0b00001.. SolvedBy=B
Difficulty=3 Sol.=0b0001.. SolvedBy=A
Why would you waste energy to create new blocks?
It's incentivised
=
You get a reward
Canonical Chain
What if two different blocks are produced with the same parent hash?
Which one should you choose?
Block 1
Block 2
Block 3
Block 3
Fork
Canonical Chain
We use "the longest" chain.
Ethereum re-organizes to a chain with the highest difficulty.
Block 1
Block 2
Block 3
Block 3
Block 4
Take away note: The latest state you see can sometimes be reverted - wait for confirmations.
Questions?
Blockchains allow for trustless transactions between multiple parties.
Building JSON-RPC APIs in Rust
By Tomasz Drwięga
Building JSON-RPC APIs in Rust
Wrocław Rust Meetup, July 2018
