The Divine Messenger and Other Architectural Ways to Bridge the Gap between Development Teams

As an Engineering organization grows and each of its autonomous development teams focuses on one corner of the product, collaborating can get challenging. Our team experienced this issue recently when we began building common workflows for managing ads across different advertising platforms. In this blog post, I share what we learned from creating a shared frontend between multiple development teams to harmonize the workflows and UI for our users.

Smartly.io has one dedicated development team per each social advertising platform, responsible for integrating them with the Smartly platform. This setup allows each team to focus on just one external API, but it can result in teams building somewhat different workflows for, say, Facebook, Snapchat and Pinterest advertising, which confuses the user.

We decided to build a common frontend which each platform-owning development team could use. Not only would it harmonize the UI, but it would also boost development speed, as teams could default to existing solutions rather than start building each workflow from scratch. While our end goal was clear, the road there wasn’t the easiest to travel. Since, for example, creating ads for Facebook requires different actions in a different order than creating ads for Pinterest, we needed to navigate the myriad of workflows built into each social advertising platform. This led us to provide the different platform teams with building blocks for creating actions, without strictly defining what those actions would be.

Relying on shared types to empower a uniform user interface

To harmonize the frontend, we set out to create some generic components that all platform-owning development teams could use to build their frontends. To help us create reusable components, we chose a shared typing for objects representing each action. We defined the type so that the action objects would always feature a set of properties needed for rendering the action. Once all the actions followed this shared base type, we were able to render the actions in different views based on the type. This allowed the platform-owning development teams to focus their energy on building the user interface for creating the action object itself and stop worrying about anything else on the frontend.

Our work didn’t end there, though. We also needed a frontend that renders the responses to the user’s actions on the UI. For example, we needed to show the user if their intended action succeeded or not. Here  shared typing helped us again, but this time we employed it on the backend.. As long as the backend service returned a response that followed the shared type, the frontend would render everything as needed,without platform-owning development teams having to build separate user interfaces to show the responses to a user’s actions.

Problems in handling everything in frontend

While creating the shared typing is easy enough we foresaw a possible issue in maintaining it. With our other backend services, we usually have a clear API which defines how to interact with the service. However, in this case, our client was the one setting the requirements for the backend. As our frontend was shared between multiple teams owning their own backends, it was easy for a team to start implementing actions that didn’t follow the shared type. This in turn would break the shared frontend components which assumed the actions and responses to follow the shared type. It was clear to us that we  needed a clear way to communicate to other development teams what their services should expect from the frontend as well as what response the frontend would expect from them.

Another issue we faced with our shared frontend was its fragility to unpredictable user behaviour. After the actions are sent to the various backends, the UI shows a spinner while waiting for their responses. However, sometimes actions can take quite a while to be processed. If the spinner goes on for too long, the user might decide to close the loading window and retry the operation in a new window to speed things up. In that case, the response sent to the frontend would never get there, rendering us incapable of indicating if the action succeeded or failed. Our solution became even more problematic when actions depended on each other, like when a campaign needed to be built before we could create ads in it. If the browser crashes or gets closed while it waits for the response, it won’t only lose the context for the ongoing actions but also halt the execution of the dependent actions.

We could increase the robustness of the system by polling the response from the frontend, but it wouldn’t help us secure the dependent actions. Previously, we had solved similar issues by always sending dependent actions in one batch to the backend and letting the backend execute the actions one by one. However, multiple backends sharing the same frontend wouldn’t allow any dependent actions across backends. Not to mention that each backend service would also need to implement identical logic for handling the dependencies.

Refining our solution: extracting shared logic into a service

We understood that shared frontend components weren’t enough in themselves to ensure a harmonized user interface across different platforms. To solve the problems laid out in the previous section, our team decided to build a mediator service that would pass the actions to the right services in the right order. In other words, the service would work as a messenger between the shared frontend and the various backend services. Aptly, we decided to call the new service Hermes, after the ancient Greek messenger god Hermes. (We first considered another Greek messenger figure, Pheidippides, but figured it would be useful if we were able to pronounce the name of our service).

So, instead of sending actions to different services, the frontend would send a batch of actions to Hermes that would then queue the actions and send them one by one to the correct backend service. 

Remembering that we needed to enforce the types of actions and responses, we decided to implement the Hermes API with our very own typescript server generator oats. We configured the types for oats and it would take care of validating that everything coming in or leaving Hermes would comply with these types. At the same time we needed to make sure the platform services would also comply with the types as we wanted to avoid mangling action data in Hermes. So we ended up extracting the type definitions in a separate library which could be shared across fronted, Hermes and platform services. The library serves as a single source of truth for the shape of actions and provides a guideline for the platform teams to build actions that can be automatically rendered on the frontend. While a shared type library already clarifies the actions schema, having Hermes validate the schema before sending that to the platform services really enforces it.

Not only was Hermes about enforcing the one user interface for handling actions, but it also allowed us to store the state of the actions in a database. When receiving a batch of actions, Hermes pushes them into a queue from where it executes the actions in order. After the execution it stores the result of the actions in a database which in turn allows a client to receive the executed actions any time, even if it lost the original polling context at some point. For the user this means that we can show the status of the actions in any browser window they open.

Having one service responsible for executing the actions also opens us new doors for improving workflow. As all the actions are going through the same pipeline, we can also make changes to the pipeline more easily. Nobody is a big fan of the loading spinners – now that everything goes through Hermes, we can relatively easily change the way results are communicated to the user by e.g. pushing them to a notification service from where they can be shown to the user as a notification. This change would require no actions from the platform teams as they can simply rely on Hermes to communicate the results. 

The result: common frontend frees focus for business logic

Sharing a frontend codebase between multiple teams can be challenging but pays off with happier developers and happier customers. A uniform user interface means smoother user experience while the shared frontend code enables teams to focus on building the business logic specific to their team instead of building workflows that could be reused. 

In the simplest case it may be enough to create abstract frontend components which are able to render items based on their shared properties. Using a typing is a great help here as it helps to communicate what sort of data structures the shared components are expecting while guaranteeing that the components work as long as their input data follows these types. If the same types can be used with backend communication, even better. In this case having a shared library for types allows reusing the types across different services and makes communicating the required types easier.

When the same frontend is used by multiple backends it is likely that it starts to involve a lot of logic for simply communicating with the backends. In this case extracting this logic into a mediator service allows better handling of dependencies between the backend calls as well as more robustness in the frontend and backend communication, as the state of the backend calls can be stored in a database. A mediator service also clearly communicates to teams what sort of communication is allowed with the frontend. As a result, teams can focus on building their business logic and be confident that the frontend is able to render their data as long as it goes through the mediator.