More Components

In this section, we will be briefly discuss about other important components which are relied by Deno.

V8

V8 is a JavaScript/WebAssembly engine by Google. Written in C++, it is also used most notably in Google Chrome and Node.js.

V8 does not support TypeScript. Instead, all TypeScript code you run in Deno are compiled to JavaScript by a snapshotted TS compiler, while the generated files are stored under .deno folder. Unless the user updates the code, only the cached JS files would be run after the initial compilation.

FlatBuffers

Flatbuffers is an efficient cross platform serialization library, developed by Google. Flatbuffers allows messages to be passed and accessed across languages without the overhead of parsing and unpacking.

In the case of Deno, FlatBuffers is used to allow intra-process message communication between the privileged and unprivileged sides. Many public Deno APIs internally create buffers that contain serialized data on the TypeScript frontend, and make these buffer available for Rust so that the Rust end could process the request. After completing or scheduling the requests, the Rust end similarly creates buffers for serialized results and sends them back to TypeScript, of which it deserializes them using files generated by FlatBuffers compiler.

In comparison to Node, which creates many V8 bindings for each privileged calls, Deno with FlatBuffers only need to expose message send and receive methods on TypeScript and Rust. This makes adding a new call much easier, and avoids direct interaction with V8.

Flatbuffers is introduced to replace Protocol Buffers in the Go prototype to avoid overhead. See this thread for more information.

Example: Passing readFile Data Through FlatBuffers

TypeScript side:

read_file.ts

import * as msg from "gen/msg_generated";
import * as flatbuffers from "./flatbuffers";
// dispatch is used to dispatch a message to Rust
import * as dispatch from "./dispatch";

export async function readFile(filename: string): Promise<Uint8Array> {
  return res(await dispatch.sendAsync(...req(filename)));
}

function req(
  filename: string
): [flatbuffers.Builder, msg.Any, flatbuffers.Offset] {
  // Builder for serializing a message
  const builder = flatbuffers.createBuilder();
  const filename_ = builder.createString(filename);
  msg.ReadFile.startReadFile(builder);
  // Filename is placed into the underlying ArrayBuffer
  msg.ReadFile.addFilename(builder, filename_);
  const inner = msg.ReadFile.endReadFile(builder);
  return [builder, msg.Any.ReadFile, inner];
}

function res(baseRes: null | msg.Base): Uint8Array {
  // ...
  const inner = new msg.ReadFileRes();
  // ...
  // Taking data out of FlatBuffers
  const dataArray = inner.dataArray();
  // ...
  return new Uint8Array(dataArray!);
}

Rust side:

ops.rs

fn op_read_file(
  _config: &IsolateState,
  base: &msg::Base,
  data: libdeno::deno_buf,
) -> Box<Op> {
  // ...
  let inner = base.inner_as_read_file().unwrap();
  let cmd_id = base.cmd_id();
  // Extract filename from serialized buffer
  let filename = PathBuf::from(inner.filename().unwrap());
  // ...
  blocking(base.sync(), move || {
    // Actual fs operation happens here!
    let vec = fs::read(&filename)?;
    // Serialize the output and send back to TypeScript
    let builder = &mut FlatBufferBuilder::new();
    let data_off = builder.create_vector(vec.as_slice());
    let inner = msg::ReadFileRes::create(
      builder,
      &msg::ReadFileResArgs {
        data: Some(data_off),
      },
    );
    Ok(serialize_response(
      cmd_id,
      builder,
      msg::BaseArgs {
        inner: Some(inner.as_union_value()),
        inner_type: msg::Any::ReadFileRes,
        ..Default::default()
      },
    ))
  })
}

Tokio

Tokio is an asynchronous runtime for Rust. It is used for creating and handling events. It allows Deno to spawn tasks in a internal thread pool and receive notifications to process the output after the task is complete.

Tokio relies on Rust Future, which is a construct similar to JavaScript Promises.

Example: Async Task Spawning

In the example of readFile above, there is a blocking function. It is used to decide whether a task should be spawned on the main thread or forwarded to the Tokio thread pool.

ops.rs

fn blocking<F>(is_sync: bool, f: F) -> Box<Op>
where
  F: 'static + Send + FnOnce() -> DenoResult<Buf>,
{
  if is_sync {
    // Runs task on the main thread
    Box::new(futures::future::result(f()))
  } else {
    // Forward the task to Tokio
    Box::new(tokio_util::poll_fn(move || convert_blocking(f)))
  }
}