Philip Guo (Phil Guo, Philip J. Guo, Philip Jia Guo, pgbovine)

Fusion: Opportunistic Web Prototyping with UI Mashups

research paper summary
Fusion: Opportunistic Web Prototyping with UI Mashups. Xiong Zhang and Philip J. Guo. ACM Symposium on User Interface Software and Technology (UIST), 2018.
Modern web development is rife with complexity at all layers, ranging from needing to configure backend services to grappling with frontend frameworks and dependencies. To lower these development barriers, we introduce a technique that enables people to prototype opportunistically by borrowing pieces of desired functionality from across the web without needing any access to their underlying codebases, build environments, or server backends. We implemented this technique in a browser extension called Fusion, which lets users create web UI mashups by extracting components from existing unmodified webpages and hooking them together using transclusion and JavaScript glue code. We demonstrate the generality and versatility of Fusion via a case study where we used it to create seven UI mashups in domains such as programming tools, data science, web design, and collaborative work. Our mashups include replicating portions of prior HCI systems (Blueprint for in-situ code search and DS.js for in-browser data science), extending the p5.js IDE for Processing with real-time collaborative editing, and integrating Python Tutor code visualizations into static tutorials. These UI mashups each took less than 15 lines of JavaScript glue code to create with Fusion.
@inproceedings{ZhangUIST2018,
 author = {Zhang, Xiong and Guo, Philip J.},
 title = {Fusion: Opportunistic Web Prototyping with UI Mashups},
 booktitle = {Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology},
 series = {UIST '18},
 year = {2018},
}

Picasso famously said, “good web developers copy; great web developers steal.” For years developers have been copying design ideas from websites that they find inspiring. However, even with a great idea that they've decided to copy, it can still be surprisingly hard to implement an interactive web app that embodies this idea. Modern web developers struggle to select and configure a motley mix of backend hosting providers and cloud APIs, then wrestle with setting up tangles of web frameworks, libraries, build systems, and dependency managers on the frontend.

What if instead of merely copying design ideas, you could simply steal interactive functionality directly from existing websites to incorporate them into your own web apps? This would allow you to instantly get the benefits of the most interesting features from millions of websites without needing access to their source code and without even installing or configuring anything.

To work toward this dream, we developed a technique that lets developers extract interactive components from existing websites and write small pieces of JavaScript glue code to fuse them together rather than implementing the desired features from scratch. Our approach does not require any access to the underlying source code or backend hosting servers of the fused apps. This eliminates the hassles of setting up and maintaining a full-stack web development environment and drastically reduces the amount of code that needs to be written. We implemented our technique in a Chrome browser extension called Fusion.

I want to illustrate how Fusion works with an example usage scenario: Imagine that Alice is a UX (User Experience) designer working at ShareLaTeX (now acquired by Overleaf!), a company that makes a web-based collaborative LaTeX editor often used by researchers when writing papers together. One day Alice comes up with a new idea to render previews of math equations inline in the text editor whenever the user highlights a relevant selection. That way, users can quickly debug and fine-tune their equations without waiting for the full PDF document to render. She wants to see how receptive users are to her new idea in order to determine whether it's worthwhile for developers on her team to put in the time and effort to implement it in the actual product.

Here's how Alice can use Fusion to quickly prototype this feature so that she can test it on potential users (click image to enlarge):

  1. She finds a web app called the Interactive LaTeX Editor that allows users to enter a line of LaTeX code into a text box and instantly see the math equation that it compiles into.
  2. She activates the Fusion browser extension and extracts two relevant panes from that page—the input text box and the rendered output display—to turn them into a self-contained widget.
  3. She writes 10 lines of JavaScript glue code to program that widget. Specifically, whenever the user makes a text selection in the browser and hits a keyboard shortcut, Fusion should copy that selection into the widget's input text box, which triggers it to render the equation in its output display. Then Fusion should pop up the widget directly below the user's current cursor. To ease this programming process, Fusion includes an inline code editor, visual element selectors, helper code snippet generators, and test harnesses.
  4. She opens her original ShareLaTeX webpage and uses Fusion to embed that widget into it as an iframe. Now she can highlight any LaTeX text selection within ShareLaTeX and instantly see a pop-up rendered preview of it.
  5. She creates a unique URL to share her prototype app with anyone who has the Fusion browser extension installed.

Using Fusion, Alice completes a functional prototype of her inline equation preview feature by simply finding an existing web app that implements a major part of her desired functionality (Interactive LaTeX Editor) and writing a few lines of JavaScript glue code to “fuse” it to her ShareLaTeX app (Figure 1). She didn't need access to the codebases or hosting servers of either web app; she just straight-up stole the functionality right from one website to put into another one! Now she can show this prototype to her colleagues to have them test it out and give her feedback on it.

In sum, what's cool about Fusion is that it lowers the barriers to prototyping web applications by letting people use existing websites not only as design inspiration but rather as programmable widgets that they can mash up with other sites. It enables anyone with basic JavaScript skills (e.g., researchers or tech-focused UX designers like Alice) to create working prototypes by stealing functionality from across the web. Picasso would be proud!


Read the full paper for details:

Fusion: Opportunistic Web Prototyping with UI Mashups. Xiong Zhang and Philip J. Guo. ACM Symposium on User Interface Software and Technology (UIST), 2018.
Modern web development is rife with complexity at all layers, ranging from needing to configure backend services to grappling with frontend frameworks and dependencies. To lower these development barriers, we introduce a technique that enables people to prototype opportunistically by borrowing pieces of desired functionality from across the web without needing any access to their underlying codebases, build environments, or server backends. We implemented this technique in a browser extension called Fusion, which lets users create web UI mashups by extracting components from existing unmodified webpages and hooking them together using transclusion and JavaScript glue code. We demonstrate the generality and versatility of Fusion via a case study where we used it to create seven UI mashups in domains such as programming tools, data science, web design, and collaborative work. Our mashups include replicating portions of prior HCI systems (Blueprint for in-situ code search and DS.js for in-browser data science), extending the p5.js IDE for Processing with real-time collaborative editing, and integrating Python Tutor code visualizations into static tutorials. These UI mashups each took less than 15 lines of JavaScript glue code to create with Fusion.
@inproceedings{ZhangUIST2018,
 author = {Zhang, Xiong and Guo, Philip J.},
 title = {Fusion: Opportunistic Web Prototyping with UI Mashups},
 booktitle = {Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology},
 series = {UIST '18},
 year = {2018},
}
Related pages tagged as human-computer interaction:
Related pages tagged as software: