Remotion 2.0 - A virtual office for your hybrid team, right on your desktop | Product Hunt

How we scripted Spotify to make room for both silence & conversations

We just launched a way to play Spotify together remotely on Product Hunt. This post covers the idea, plus some technical details of how we built it.

The idea: Comfortable silence leads to unexpected, meaningful conversations

Would you ever have a nuanced discussion at a football game? Probably not. For a deep discussion, eliminate distracting noise. But, who hasn’t suffered as every second of silence at a Zoom happy hour felt like an hour? Remote silence needs background noise to feel comfortable.

Although these scenarios are opposites, together they’re magical: Comfortable silence leads to unexpected, meaningful conversations. It happens naturally in person: Perhaps working around a table, or maybe walking to a meeting. On the other hand, remote teams are missing precisely those conversations.

So we set out to use background music to create calls where long gaps in conversation would feel natural.

The feature: Shared music + voice chat, long-running in the background

We started with Remotion rooms, which we’d already built to be lightweight so you can keep them open in the background while you work. Next, we added some generic music. We quickly heard two problems:

  1. Power users wanted to share their personal music, which was usually in Spotify.
  2. Music + voice was messy: It was a hassle to get everyone to wear headphones and mute when not speaking. And the music got in the way of talking.

So we came to our ideal spec: What if you could listen to music in Spotify, with coworkers? And whenever someone spoke, the ambient music automatically faded to allow for conversation, then returned when conversation died out? It would be like having your own ultra-attentive DJ Butler to help you remain relaxed and productive.

How we built it: Scripting bridge, music fading, and a boatload of reactive code

Sharing Spotify music

There were a few incredibly different ways we could approach this:

Pros Cons
Peer-to-Peer audio streaming
  • Only the streamer needs Spotify
  • Poor audio quality
  • No local volume control
  • Cannot leverage Spotify app for playlists etc
  • Very complicated to build music fading
Play in Spotify using Spotify Server APIs
  • Playback in Spotify app, offering user volume control, playlists, quality, etc
  • Doesn't require Spotify macOS
  • More examples of this on the internet
  • Laggy
  • Cannot build music fading
  • Requires authing with Spotify
Play in Spotify using AppleScript APIs
  • Playback in Spotify app, offering user volume control, playlists, quality, etc
  • Immediate
  • Can build music fading
  • Requires desktop app
  • Requires granting Remotion.app Automation permission for Spotify.app
  • May not generalize to other platforms

In the end, we opted for what we thought would be the best user experience. Using AppleScript APIs.

A brief digression into what you can do with Scripting Bridge on macOS

If you’re trying to figure out what you can do with Scripting Bridge, here’s a way to inspect what apps support:

If the app you’re inspecting has support, this will generate a header file for you named something like “spotify.h”. Here’s an excerpt of that:

Interestingly, most music apps seem to support a common set of commands—we haven’t confirmed, but we think this is to enable common macOS conveniences, like play/pause, track display, etc.

Fading music to avoid feedback and distractions

When we started on the feature, we heard a bunch of feedback around music being distracting. Sometimes it was echo or feedback, other times it was too loud. Our first attempts to clean this up worked well for users on speakers, but were too aggressive for others wearing headphones. It seemed complex!

Ultimately, we realized that we could boil things down to a relatively simple spec, paired with some overridable smarts to determine whether the user was on speakers or headphones:

I'm wearing headphones I'm using speakers
Someone else is talking Fade the music so I can hear them Fade the music so I can hear them
I’m unmuted but not talking No need to do anything Fade the music to prevent echo
I’m unmuted and talking Fade the music so I can hear myself No need to do anything because we already faded the music above

Additionally, people speak differently. How do we account for inconsistent speaking volumes and vocal cadence and make sure the music never raises while someone is still speaking? We realized we needed to normalize the input levels of your voice and other speakers, and make sure to debounce events to make sure conversation had truly died out before raising the volume. We did this by making sure that the input volume for the current user (me) and other users (everyone else) stayed above a certain threshold.

We then use Combine to watch for changes in the status as we get new mic events from the audio stream:

This allowed us to detect if we should lower the music volume based on the mic inputs. Ship it!

Debouncing and delaying the unfading

Close, but not quite done. Turns out, we’d created a pretty poor user experience. We heard about it quickly:

One issue is that there are now three volume levels for music that you can experience in a very short amount of time: normal, slightly dipped when someone else is unmuted, extremely dipped when you are unmuted. Another thing is that if you normally like to have the music a little louder, the dipping-on-voice becomes really distracting. Altogether it’s very distracting and, like I say, we end up pausing the music which sort of ruins one of our favourite Remotion features.

We needed to do two more things:

  1. Make sure we were properly debouncing events to make sure we weren’t responding to duplicate statuses and being overly ambitious with lowering the volume, and
  2. Introduce an artificial delay after all the mic levels drop below their thresholds to make sure the conversation has died out. We recognized we do this naturally when we are speaking. Folks of most cultures tend to listen to someone speak, and only respond after they hear a long enough delay since the last word their partner spoke. Here is how we emulated this:

One of the interesting (and debated!) decisions here was to use both Swift Concurrency and Combine in the same code. Although it isn’t the purest implementation, we found it to be the most readable one we could write. What do you think? (My email at bottom of post.)

Future posts: How we’re rewriting Remotion with Swift Concurrency and Combine

We wrote the systems shown quickly (Swiftly?) using two tools: Swift Concurrency and Combine. These help us write clean, performant code with as little imperative state management as possible. In fact, this was a complete rewrite our first version, which was written imperatively and was incredibly difficult to stabilize. Bugs and edge cases galore.

At Remotion we’ve now rewritten many things this way to great success. If you’re interested in future blog posts about that and how we’re building Remotion, sign up below. Also, if you have any topic requests, drop me a line! I’m zak at remotion dot com.

Thank you!

The case for virtual coworking: build a connected remote culture.

Regularly coworking with your hybrid or remote team can help you build the social cohesion that makes work feel less like work.

Here are the biggest reasons we think virtual coworking is an effective way to create a close-knit remote culture:

1. It fosters casual conversations.

Building a connected remote culture is all about fostering 1:1 or small group organic conversations. Virtual coworking makes space for those conversations. When you spend time together outside of agenda-driven meetings, spontaneous chats naturally occur, as they would in a traditional office.

2. It's more inclusive than scheduled social events.

It can be draining for introverts to have to participate in scheduled, purely social conversations. Virtual coworking allows the team to spend time together and occasionally chat without having to constantly be "on," making it more inclusive for introverts and extroverts alike.

3. It's easy to say yes to.

Purely social events are important, but if your remote team is busy or on a tight deadline, it's tough to find the time for social chats without it feeling like an obligation. Coworking is much easier to get your distributed team onboard with because it doesn't take time away from getting work done.

4. It improves remote collaboration.

Coworking can lead to unblocking and shorter feedback loops and stronger remote collaboration. Quick questions get answered easily and in the moment, without a having to schedule a meeting or go back-and-forth in messages. Coworking also builds peer accountability.

5. It's scalable.

Coworking works for teams of all sizes and is a great way to scale your remote culture as your team grows. It's helpful to create opportunities for teammates from different functions to get to know one another.

6. It creates shared momentum.

Virtual coworking helps remote workers for the same reason you might get a membership at a traditional coworking space: the feeling of togetherness is motivating!

Get started with virtual coworking: choose the type most aligned with your priorities.

It takes intentionality to make virtual coworking feel natural and energizing enough to stick—it's not as simple as leaving a Zoom call open all day.

Here are a few of the ways we've set coworking up for our team. We recommend choosing one to start with. If it works, make it routine and experiment with other types from there.

Best practices for virtual coworking.

Keep group sizes small.

Limit your work sessions to 4-6 people to minimize distraction and help make introverted teammates comfortable chatting.

Signal boost coworking.

Set a norm of letting the entire team know when you're hopping into a coworking room or session.

Make it routine.

Once you've figured out what kind of coworking works for your team, make it a regular, opt-in event. Set up a recurring calendar event to do it at the same time each week to maximize the impact.

Set expectations ahead of time.

When you're first introducing coworking to your remote team, share what you're imagining in your calendar invite and at the top of work sessions to get everyone on the same page. For example:

Let's try virtual coworking! We'll work independently on our own projects with our cameras off, but we'll share virtual space and listen to music together — like we might work side-by-side at a physical office.

Listen to music together.

Play music while you work in a virtual room to create a shared environment and add a little bit of personality to your virtual coworking session.

Set up Coworking Rooms in Remotion.

Most of the above is doable with any video chat app or virtual office, but much easier with Remotion—which we designed with a lightweight, smooth coworking experience in mind. Remotion is the perfect virtual coworking platform—easily set up virtual rooms that your teammates can hop into for different styles of coworking.

While Remotion's virtual workspace is free to use with your remote team, if you're curious about joining a virtual coworking community built on our platform—check out Swift Remote Studio for iOS, Mac, and Swift developers.

Want to try coworking in Remotion? Get free access today .

We'd love to hear how coworking goes for you, or what practices you've found helpful on your team — let us know @remotionco on Twitter.

Want to try coworking in Remotion? Get free access today.