WYSIWYG Reports

sports_esportsExample report

At Weights & Biases, I led the design and development of the Reports project, a mission to make it easy to share your machine learning research with your team and the world. As of July 2021, more than 500 reports are created every week.

The Problem

When I joined W&B, reports already existed, but they were designed for only one purpose: sharing progress updates internally with teammates. As a result, it was designed for quick, one-off, unpolished notes.

before

We saw an opportunity to enable researchers to quickly publish their research to the world using the reports that they were already creating for internal use. Some users were already trying to using reports this way, but were having a hard time. To create a good publishing experience, we needed to solve 3 problems:

  1. Make reports visually clean, readable, and mobile friendly.
  2. Create a delightful longform text editing experience.
  3. Make reports collaborative.

Cleaning up the look

First, we needed to fix the low hanging fruit. A clear problem was that there were no separate "edit" and "view" modes, so all viewers would see all the editing widgets even if they didn't have permission to use them. We solved this by creating a view mode where the editing features were hidden and the boundaries between panels were less pronounced.

constructionTODO: Add image

Next, we tackled a more fundamental problem. Reports were initially just collection of panels. More precisely, a report was a collection of sections, and a section was a collection of panels along with a dataset to hydrate the panels. A panel could contain either a chart, a table, or text.

This worked for sharing quick notes with coworkers, but was an unfriendly experience for readers without context. Readers expect content to have a linear flow so their eyes always know where to go next. We noticed that to accomodate this, many of our users were creating sections that only contained full-width text panels to closer resemble a blog post.

There were several problems here:

  1. The text was too wide. Text is only readable at 45-75 characters per line.
  2. The panel borders were disorienting.
  3. Creating a new section with one full-width text panel was a 5 click + 2 drag process.

We tried hacking around problems 1 and 2 by setting a max-width on the text inside panels and removing borders specifically for full-width text panels, but that created surprising and confusing behavior. Ultimately we needed text to be outside of panels.

So our next approach was to make text "first-class"; in between any two panel sections, you could insert text, and this text would have a smaller max-width than the panel sections.

constructionTODO: Add image

This essentially gave reports the look we wanted. However, underneath the veneer, the editing experience was still a mess.

Going WYSIWYG

Text editing was initially all Markdown based. In many ways, this made sense. Our users were very technical, so they were mostly familiar with Markdown, as it's the language of popular tools like Github. And historically, technical users have expressed a strong preference for Markdown over WYISWYG.

However, Markdown wasn't made for complex documents with rich interactive visualizations. We had worked around this by having Markdown sections and panel sections be logically separate. Unfortunately this had frustrating UX problems:

Ultimately we decided that WYSIWYG was the only way to get the user experience we really wanted. Historically, customized WYSIWYG editors have been extremely challenging to build. But after surveying the open source landscape and asking my friends at other companies what they used for their text editors, it seemed like we had better options than fighting with contentEditable directly. The two most promising libraries were Prosemirror and Slate. I did a long writeup on their similarities and differences, but here's the gist:

It was a tough decision, but I decided we should take the risk and run with Slate. Part of my justification was that I had already read Slate's source code in its entirety because it was so small and elegant. So whatever bugs we encountered, I was confident I could deal with it, forking the library if need be.

In retrospect, it's still unclear whether this was the right call. We ran into numerous bugs and performance problems that don't seem to be present in ProseMirror, and the library maintainers have not been patching them quickly. However, we were able to work around these problems with relatively clean and modular code, and we learned a lot about the internals of Slate in the process. Axel Setyanto, who gave reports vital performance improvements, has said, "Reports has become my favorite product to work on". So while it was a struggle, we don't regret it.

Features

Slate truly comes as a blank slate; it gives you the ability to edit plain text and insert new paragraphs, but all rich text features like bold and headings are on you. Open source Slate plugins were scarce and outdated, so we ended up implementing everything ourselves, including:

Notion and Dropbox Paper were our biggest inspirations in the Reports design. We loved the way they auto-convert Markdown formatting, so we borrowed that idea in Reports:

We also borrowed from Notion the slash menu, used for adding anything that doesn't have a Markdown shortcut, or anything you don't remember the Markdown shortcut for:

Visualizations

Admist all the exciting rich text editing features, it was important to not forget the heart of reports: the visualizations. WYSIWYG opened up the potential to interweave text and visualizations in ways that were previously impossible, so I wrote a document detailing some ideas on that. However, our CTO Shawn had a fundamentally new architecture for visualizations in the works, so the future was uncertain to the point where it wasn't worth making drastic changes to the behavior of visualizations. So our short-term goal with visualizations was simple: make them not worse than before.

I did end up making one improvement to visualizations that I felt was necessary: undo/redo. The rest of the document supported undo/redo, so I felt it would be surprising if visualizations didn't, even though they never did before. The process of integrating visualizations with Slate's undo/redo system deserves its own blog post, but I'll just say it wasn't pretty.

Maintaining backwards compatibility

Because reports had many active users from the start, we chose to maintain backwards compatibility with very revision. We never released a "Reports version 2" that made old reports obsolete.

This presented a challenge because the changes we made were so sweeping. I won't go into the technical migration details because they're too messy. But I'll talk briefly about how we transitioned from Markdown to WYSIWYG.

First, we supported Markdown blocks within the WYSIWYG document so that users who preferred Markdown could continue to use it.

Next, my teammate Altay Guvench created a function to automatically convert Markdown into our custom JSON structure. Rather than applying it automatically to every report, we decided to make it opt-in. We migrated old reports into our new structure, but left the Markdown sections as Markdown blocks. We then displayed a banner message above old reports asking if the authors would like to auto-convert the content into WYSIWYG. Alternatively, users could convert their Markdown blocks one at a time.

Going collaborative

There exists a third-party plugin in the Slate ecosystem called slate-collaborative that essentially provides drop-in realtime collaboration. However, our effort to solve collaboration unfortunately preceded our usage of Slate. Users were clawing for collaboration from the get-go because reports were initially extremely restrictive; only the author of a report could edit it at all.

Realtime collaboration was out of the question because it would have taken too long. It requires certain architectural decisions to be made up-front, so we would have needed a complete rewrite to support it. Asynchronous collaboration was a more prudent problem to solve.

Still, asynchronous collaboration turned out to be more complex than I had anticipated. What do you do when a user tries to edit a report while another user is already editing it?

The simplest possible solution would be to lock everyone else out while a user is editing a report. This is the solution employed by Webflow. But because we used Webflow for our marketing site, we knew that this solution creates a terrible user experience. The problem is that it's too easy to accidentally block someone from editing the site, because you may forget that the tab is still open and Webflow has no way to tell whether you're really done editing.

After some deliberation, I decided to adopt a git-like system. When you edit a report, you create a draft copy of it private to yourself, like a local clone of a repository. Each user with edit access can have their own private draft of a report. As you edit a private draft, it's autosaved so that you can make long-running edits. When you're ready to publish your changes, you can merge into the "master" report, and you'll be informed of any potential merge conflicts.

Later on, after we implemented WYSIWYG, I quietly tested out slate-collaborative to make a proof-of-concept of realtime collaboration. It drew delight from the team but probably won't see production for awhile.

prototype

Controversial design details

Default text width

Text is most readable at less than 70 characters per line. But visualizations want to be wider than that. Should we align the text to the visualizations, or should we make the text readable at the cost of awkward alignment when there's little text?

We initially opted to default to wide text since longform text was uncommon. But as we grew Reports into more of a publishing platform, we switched the default text width to be readable.

Drag handle on the right or left

[details tbd]

Serif vs. Sans Serf

Before I touched them, reports were entirely sans-serif. In my first design pass, I changed them to be entirely serif. Serif made more sense to me because machine learning researchers are used to reading research papers in serif, and serif fits better with LaTeX.

Over the next couple months, a few coworkers told me they preferred sans-serif because it's more hip and modern. I understood where they were coming from; serif feels older because it's more common in print than on websites. However, on screens, serif is actually a very modern thing. A decade ago, screens were too resolution for serif to be readable, which is why sans-serif used to be recommended.

I wasn't too attached to my serif preference though, so I swapped reports back to sans-serif to see how others would react. Suddenly the whole company, including engineers who had never previously expressed design opinions, began fervently debating the merits of serif and sans serif. Bike-shedding is normally considered to be a bad thing, but I delighted in watching eng

Closing takeaways

Thanks to libraries like Slate, making a collaborative document editor with rich interactive visualizations is much easier than it ever was before. Thank you to Ian Storm Taylor for writing such a lovely library and giving it away for free. And thank you to Axel Setyanto, Altay Guvench, Kat Shuford, Haruka Tabuchi, and others for continuing to maintain Reports.