Visualizing our Data

Visualizing our Data Interactively

• Interactivity allows on-the fly parameter tuning
• ... it also allows for exploration
• Reveals interesting science!
• Allows science to be accessible
  • to non-scientists
  • to those that don't identify as "hackers"
  • to those that are new to the data

Tools Enabling Interactivity Make Science Accessible

Using Jupyter Widgets out-of-the-box With yt

Rendering an Image

Rendering an Image

Rendering an Image

Rendering an Image

Rendering an Image

When might this become cumbersome?

$T_{server}=t_{signal}+t_{serialization}+t_{image\: calc}+t_{pull} + t_{display}$

multiply by $n$ interactions

What if

Pushing Data to the Browser Makes Sense as

$\begin{align}T_{client} &< T_{server}\end{align}$

\[\begin{align}n \cdot t_{client} + \frac{data_{dataset}}{r} &< n \cdot \bigg[t_{server} + \frac{data_{image}}{r}\bigg]\end{align}\]

where
\[ \begin{align} n &= \mbox{the total number of interactions} \\ r &= \mbox{the data transfer rate} \\ t_{client} &= \mbox{the total time to compute an image on the client} \\ t_{server} &= \mbox{the total time to compute an image on the server} \\ \end{align} \]

How might this happen?

\[\begin{align}n \cdot t_{client} + \frac{data_{dataset}}{r} &< n \cdot \bigg[t_{server} + \frac{data_{image}}{r}\bigg]\end{align}\]

↑ $n$

↑ $data_{image}$

↓ $t_{client}$

Using yt-canvas-widget

What does this mean?

With a single, fixed, up-front cost we eliminate the requirement to transfer n images between the server and client

This is useful in some, but not all cases

How it works: the first render

How it works: the first render

How it works: the first render

How it works: the first render

How it works: the first render

How it works: changing the colormap

How it works: changing the colormap

How it works: navigating with canvas interactions

How it works: navigating with canvas interactions

How it works: navigating with canvas interactions

How it works: navigating with canvas interactions

How it works: navigating with canvas interactions

Why Webassembly?

• fast
• wasm objects work with javascript
• sandboxed memory environment
• reduces $t_{client}$



Why Rust?

• safe
• performant
• welcoming

wasm-bindgen : the rust side

                                                             
                             #[wasm_bindgen]
                             pub struct VariableMesh {
                                 px: Vec,
                                 py: Vec,
                                 pdx: Vec,
                                 pdy: Vec,
                                 val: Vec,
                             }
                             
                             #[wasm_bindgen]
                             impl VariableMesh {
                                 pub fn new(
                                     px: Vec,
                                     py: Vec,
                                     pdx: Vec,
                                     pdy: Vec,
                                     val: Vec,
                                 ) -> VariableMesh {
                                     VariableMesh {
                                         px,
                                         py,
                                         pdx,
                                         pdy,
                                         val,
                                     }
                                 }
                             }
                             

wasm-bindgen : the js side

                             /* tslint:disable */
                             import * as wasm from './yt_tools_bg';
                             
                             function passArrayF64ToWasm(arg) {
                                 const ptr = wasm.__wbindgen_malloc(arg.length * 8);
                                 getFloat64Memory().set(arg, ptr / 8);
                                 return [ptr, arg.length];
                             }
                                                             
                             export class VariableMesh {
                             
                                             static __construct(ptr) {
                                                 return new VariableMesh(ptr);
                                             }
                             
                                             constructor(ptr) {
                                                 this.ptr = ptr;
                                             }
                             
                                         free() {
                                             const ptr = this.ptr;
                                             this.ptr = 0;
                                             wasm.__wbg_variablemesh_free(ptr);
                                         }
                                     static new(arg0, arg1, arg2, arg3, arg4) {
                                 const [ptr0, len0] = passArrayF64ToWasm(arg0);
                                 const [ptr1, len1] = passArrayF64ToWasm(arg1);
                                 const [ptr2, len2] = passArrayF64ToWasm(arg2);
                                 const [ptr3, len3] = passArrayF64ToWasm(arg3);
                                 const [ptr4, len4] = passArrayF64ToWasm(arg4);
                                 return VariableMesh.__construct(wasm.variablemesh_new(ptr0, len0, ptr1, len1, ptr2, len2, ptr3, len3, ptr4, len4));
                             }
                             }
                             

wasm-pack

Demo: Pop II Prime

Limitations of this Widget:

• requires a 2d data slice
• potential limitation for data sizes
• limited functionality
• requires maintenance in multiple package ecosystems and languages
• still early in development (yt-tools @v0.2.0, yt-pycanvas @v0.1.7)
• the tools are new

Moving Forward

• Broaden the functionality of the widget
  • Linking with data units and values
  • Allow for different dimensionalities
  • Leverage traitlet linking for multiple field types
• Build out cross-functionality with yt
• Restructure widget to enhance performance

Links

• rust-yt-tools repository
• yt-canvas-widget repository
• Data and paper for widget demonstration
• data slice and data projection used in the demo
• Luiz Irber's poster on creating rust-backed python libraries (Scipy 2018)
• Rust + WebAssembly news

Installing the widget:

we are on PyPI

doesn't require Rust

doesn't requre you to compile to webassembly

                            pip install yt_pycanvas
                            jupyter nbextension enable --py --sys-prefix yt_pycanvas
                        

Installing the widget (development**):

requires Rust (nightly) / node / npm, and compilation to webassembly

                            git clone https://github.com/data-exp-lab/yt-canvas-widget
                            git clone https://github.com/data-exp-lab/rust-yt-tools
                            cd ./rust-yt-tools
                            wasm-pack init --scope data-exp-lab
                            cd ../yt-canvas-widget/js/
                            npm install --save ../../rust-yt-tools/pkg/
                            npm install
                            cd ../
                            pip install -e .
                            jupyter nbextension install --py --symlink --sys-prefix yt_pycanvas
                            jupyter nbextension enable --py --sys-prefix yt_pycanvas