Moving Data Between Layers in Synapse: Avoiding Mistakes and Fixing Catastrophes

by savage | 2026-04-13

"To err is human, to work in a forked view, divine" - Alexander Pope

One of Synapse's most powerful features is the ability to segment data using views and layers. There are a range of potential uses, from providing analysts with a "scratch space" in which to work or a new analyst a "training ground" in which to familiarize themselves with Synapse, to enforcing data privacy protections. As analysts, we often need to move data between layers, such as when merging recently completed analysis down from a research layer into an underlying production layer. In this blog, we'll discuss best practices for moving data between layers safely and efficiently, and, because accidents happen, we’ll also discuss ways to recover misplaced data.

Best Practices for Moving Data Between Layers

The Vertex Project recommends the following best practices for safely moving data between layers in Synapse, and protecting data from unintentional changes and merges:

Fork and Merge Workflow

We strongly recommend that Synapse users minimize risks to their data by adhering to what we call the fork and merge workflow. As analysts using this workflow, we’ll begin a new research task by forking a view to create a scratch space, before completing our work within that forked view, reviewing any changes that we’ve made, and then merging the approved changes back down into the underlying layer. Working in a forked view reduces the likelihood that we’ll unintentionally modify existing data, and creates an opportunity for us to review our work before propagating any changes downward. Throughout this process, we’ll have full visibility into the underlying layer and can take advantage of prior research and analysis to inform our current work.

Setting Permissions

An organization can use permissions to enforce data boundaries and restrict which users can move data between layers. Permissions within Synapse can be as simple or as granular as needed. For example, an organization may prevent a newer user from merging data entirely, or choose to prevent all users from merging a particular tag, certain nodes, or nodes with a particular tag into a production layer.

Protecting a View

When forking a view, we can use the "protect" setting to prevent ourselves or another user from merging down or deleting the view and data within the associated layer. We can mark a view as protected either through the Fork View modal or the View Configuration page in the Workspaces Tool. When the "protect" setting is applied, the merge and delete icons in the Top Bar are greyed out so no one can accidentally click them.

_images/protect_view.webp — *Setting the "Protect" toggle on when forking a view*

_images/workspace_protect.webp — *Setting a view as protected in the View Configuration panel in the Workspaces Tool*

_images/topbar_protect.webp — *Once a view is protected, the merge and delete icons are greyed out*

While this setting prevents anyone from using the merge and delete icons in the Top Bar to merge or delete the view as a whole, be aware that it is still possible to use the merge and delnode Storm commands to selectively merge or delete data within the view.

Enabling Quorum

Synapse's Quorum feature offers another layer of protection against accidental merges. When Quorum is enabled on a view, we’ll need to create a merge request for our fellow analysts to review and approve prior to merging that view down. A specified number of reviewers will evaluate the changes that we’ve made and vote whether or not to approve our merge request. The reviewers must reach consensus before Quorum will propagate the changes down into the underlying layer. As with the protect setting, we can still incrementally merge selected nodes using the merge Storm command, even with Quorum enabled.

_images/quorum.webp — *Enabling Quorum through the View Configuration panel in the Workspaces Tool*

What to Do When Accidental Merges Happen

Unfortunately, accidents can still happen, even when adhering to these best practices. Fortunately, there are several options available in the event that we’ve accidentally merged data. The best path for recovering that data will depend on the amount of data that we’ve merged, where we’ve merged the data to, and what permissions we have in the affected layers.

Assessing the Damage

First things first: how much data did we merge, and what layer did we merge it into? If we’re unsure of how to answer either of these questions, then we might turn to Synapse Metrics. Synapse Metrics is an Advanced Power-Up that tracks layer activity and records changes that users have made. A different user may be associated with the changes, depending on whether the merge occurred through a user clicking the merge icon in the Top Bar, running the merge Storm command, or through approval in Quorum.

If the merge took place recently, we can attempt to identify the affected nodes by using the following Storm query to lift all those created in the underlying layer within the last five minutes:

.created@=(now, -5 mins)

However, this may not be practical in a layer that sees many recent changes due to other users or active automation.

An alternative option would be to use the Metrics Tool to view a breakdown of recent changes made within the Synapse instance. Filter by scope and time range to identify what changes took place in a specified write layer within an identified time span, then switch to the Console Tool and use the metrics.nodes.byuser and metrics.edits.byuser Storm commands for a more precise breakdown of what data moved and to which user the changes are attributed.

Understanding the scope of an accidental merge is the first step toward correcting it, as the appropriate recovery path will depend on how much data moved, and where it landed. In some cases, for example, the fix might be as simple as removing tags that were pushed down. However, an unintentional merge that subsequently launched a trigger and resulted in additional changes would likely necessitate a more involved cleanup.

Understanding Your Permissions

Our recovery options depend significantly on where the data landed. For example, if we’ve accidentally merged data into a layer that we own or otherwise have administrative access to, then we have necessary permissions to clean up the data ourselves. Otherwise, we’ll likely need assistance from a user with the appropriate permissions.

Choosing Your Approach

There are two options for correcting the accidental merge: using Storm edit syntax to delete the erroneous data from the new layer before recreating it in its original layer, or using the movenodes Storm command to relocate the nodes.

Using Storm Edit Syntax

For more minor merge mistakes - such as having accidentally merged tags or light edges down to an underlying layer that already contains the associated nodes - we may opt to use Storm edit syntax to delete the tags or edges and then recreate them in the original layer. Tags and light edges are designed to be flexible and easy to apply, change, and delete. Unlike nodes, they cannot themselves carry tags and do not have properties, giving them a somewhat smaller footprint in the graph and making them easier to modify or delete when needed. Tags do have a corresponding syn:tag node, which we may also want to relocate, especially in the event that the node represents an internal tag pushed to a production or less private layer.

Using the Movenodes Command

For everything else - there is the movenodes command. The movenodes Storm command is the best option to address merge errors that require relocating nodes. Rather than using Storm edit syntax to delete and recreate data, we can use movenodes to relocate data from one layer to another, provided that the layers exist within the same layer stack.

We’ll specify which nodes to relocate and the movenodes command will move the associated "storage nodes" - that is, the node and associated data such as its primary property, secondary and universal properties, and tags that exist within that layer. If the node is the "left hand" of two or more nodes connected by a light edge (such as n1 in n1 -(*)>n2), then the movenodes command will relocate both the node and the associated light edge, preserving the modeled connection between the two nodes.

By default, the movenodes command will only move specified nodes up one layer. If we need to relocate nodes up several layers from their current location, we can specify the source and destination layers to move the nodes directly there, rather than needing to rerun the command to move the nodes up through the intermediary layers. In the event that the nodes have different property values in different layers, we can use the precedence option to identify by layer which value should move along with the node.

Movenodes in Action

As an example, let’s say that I have three ps:contact nodes in the bottom layer of a three layer stack:

_images/base.webp — *Three ps:contact nodes with :name, :loc, and :email property values set*

There are several different property values for these nodes in the layer immediately above this one, as seen below:

_images/middle_fork.webp — *Three ps:contact nodes with different :loc and :email properties*

When viewing these nodes from the top layer, I see the property values from the middle layer displayed in the Results Panel. However, if I were to select and view one of the nodes in the Anatomy tab of the Details Panel, I would see where the node and the different property values are located within the layer stack:

_images/anatomy.webp — *The Anatomy tab in the Details Panel displaying in which layers the node, property values, and tag reside*

I want to move these three ps:contact nodes from the base layer to the top layer in the stack, and retain the property values that are set in the base layer, rather than keeping those set in the middle layer.

The Storm query shown below assigns the three layers (base, middle, and top) to variables and then lifts the ps:contact nodes tagged with #move and pipes them to the movenodes command. The query lists the base and middle layers as the source layers, identifies the top layer as the destination layer where the nodes should be moved to, and then uses the "precedence" option to specify that the property values from the base layer should be given precedence (over those in the destination and middle layers). The command uses "apply" to execute the move - without this, Synapse would print out what the command would do, without executing the changes.

_images/command.webp — *Movenodes Storm command specifying source and destination layers, and property value precedence*

Executing the command moves the three ps:contact nodes to the top layer in the stack, where they will have the property values from the base layer. These nodes are now only visible through a view that contains the top layer in the stack - they are no longer visible in views that only contain the base or the base and middle layers. Now, if I were to look at one of the nodes in the Anatomy tab of the Details Panel, I’d see that the node, property values, and tag all exist within the top layer, as seen below: