Limited Series Podcast

TLDR: Key Takeaways

• The best analyst tools aren't built for analysts; they're built with them. Close, ongoing collaboration between analysts and engineers catches edge cases and workflow realities that traditional requirements-gathering can easily miss.
• Better tooling gives analysts more time to actually analyze. Automating data wrangling, normalization, and repetitive technical work allows analysts to spend more of their time applying judgment and expertise.
• Fast iteration makes better software possible. Frequent releases allow modeling decisions, usability issues, and newly discovered requirements to be corrected quickly rather than locking users into imperfect decisions for months or years.
• Good collaboration requires expertise without ego. Analysts don't need to know how to engineer every solution, and engineers don't need to understand every analytical nuance. Both sides need enough humility to recognize where their expertise ends and someone else's begins.
• The analyst-engineer divide doesn't have to stay a divide. Synapse power users increasingly occupy the space between the two disciplines: analysts can automate workflows and write Storm, while engineers develop a deeper understanding of data modeling and analytical use cases.
• Not every technically possible feature is worth building. Engineering still requires tradeoffs between usability, performance, complexity, and the frequency with which a particular capability will actually be used.
• Synapse's evolution reflects the collaboration behind it. Over ten years, the platform has grown from a largely command-line research tool into a broader platform supporting everyone from point-and-click users to analysts building their own queries, automations, and Power-Ups.

When Analysts and Engineers Speak Different Languages

On paper, analysts and engineers should have plenty in common. Both disciplines depend heavily on logic, understanding inputs and outputs, and solving complicated problems. In practice, however, the details of those disciplines can create a significant divide.

Analysts understand how investigations actually unfold: the messy data, strange exceptions, ambiguous relationships, and questions that can't always be reduced to a clean either/or answer. Engineers understand what's happening under the hood and how those analytical requirements can (or can't) translate into reliable, performant software.

Bridging those perspectives requires more than handing engineers a list of requirements. It requires communication, collaboration, and a willingness to understand how the other side approaches the problem.

At The Vertex Project, that relationship has existed from the very beginning. Synapse's earliest development paired engineers building the platform with analysts actively testing it, identifying problems, and helping shape how it needed to work.

Why Edge Cases Matter

Software development naturally tends to focus on primary use cases. But analysts have a habit of finding things that exist around the edges.

A simple question of whether something should work one way or another frequently gets an analyst's favorite answer: both. That's because real-world data rarely fits perfectly into predefined categories. Even within a single use case, analysts need to capture nuance, uncertainty, unusual relationships, and exceptions. Those "edge cases" may seem marginal from an engineering perspective, but they can be critical during an actual investigation.

Having analysts directly involved in development helps identify those cases early and gives engineers the context needed to decide whether the platform should accommodate them and if so, how.

Building Software With Users, Not Just Requirements

One of the recurring themes throughout the conversation is the importance of proximity to the people actually using the software. Traditional development can involve requirements moving through multiple layers before reaching engineering. By the time something gets built, the original problem may have been simplified, misunderstood, or stripped of important context. The Vertex Project takes a much more direct approach.

Internally, engineers can work alongside analysts who represent many of the platform's real-world use cases. Externally, Synapse users can interact directly with analysts and engineers through the Vertex community Slack. That means a user encountering a modeling question can talk to someone who understands the analytical implications, while someone experiencing a technical issue may end up speaking directly with the engineer responsible for that functionality.

Let Analysts Analyze

A significant portion of analytical work has historically been consumed by something that isn't really analysis at all: data wrangling. Analysts have to find information across different systems, move it somewhere useful, normalize it, combine it with other information, and figure out how all of those pieces relate before they can begin answering the actual intelligence question. Good tooling changes that equation.

When software handles more of the repetitive work involved in collecting, structuring, and connecting data, analysts can spend more of their time on the part of the job that actually requires their expertise: analysis. That's one of the fundamental problems Synapse was originally designed to address - not replacing analytical judgment, but removing unnecessary friction around it.

The Reality of Messy Data

Different providers can represent the same underlying concepts in completely different ways. Relationships may lack context. APIs may expose data without clearly explaining what it means. And sometimes even the organization producing the data can't provide the additional clarity an analyst would ideally want. Building integrations around those sources therefore becomes a collaborative exercise.

Analysts help determine what the source appears to mean and what context users need. Engineers determine what's technically available and how it can be represented consistently within the platform.

Sometimes there isn't a perfect answer. The goal becomes representing the available information as accurately and transparently as possible without implying certainty the source doesn't provide.

Good Engineering Is Sometimes Invisible

Some of the most successful software design is also the easiest to overlook. When an interface is intuitive, a workflow is frictionless, or a query simply performs the way someone expects it to, users don't necessarily stop to think about the engineering that made it possible.

That makes analyst feedback especially valuable. Sometimes an analyst can identify exactly what's creating friction. Other times, an engineer watching how someone uses the platform notices an awkward workaround the analyst has simply learned to live with. Close collaboration means improvements don't always have to begin with a formal feature request. Sometimes they begin with an engineer asking: Why did you need to do it that way?

Meeting in the Middle

Over time, the line between "analyst" and "engineer" can become surprisingly blurry. Synapse analysts can begin with straightforward Storm queries and gradually move into increasingly sophisticated automation, ingestion, and development tasks. Power users can build macros, write Power-Ups, and interact directly with more advanced parts of the platform. At the same time, engineers working closely with analysts begin internalizing analytical concepts and modeling principles.

That middle ground is valuable because it reduces the amount of translation required between the people defining a problem and the people building the solution.

From Research Project to Analyst Platform

The Synapse of today looks very different from the platform that existed ten years ago. Early versions were largely command-line driven, requiring users to maintain much more of a mental map of the data model themselves. Over time, Synapse expanded to include a full user interface and tools including Spotlight, Stories, the Storm Editor, the Learning Tool, and Rapid Power-Ups. The community around the platform changed along with it.

What began as a research project known to a relatively small group of people evolved into a platform that users now bring from organization to organization - sometimes because a former coworker introduced them to it or because it became their preferred way to approach threat intelligence analysis.

For the team, one of the most gratifying parts of that evolution has been watching users reach what thesilence describes as the "light bulb moment": the point where Synapse stops looking like another place to store intelligence data and starts changing how they think about analyzing it.

Building for Point-and-Click Users and Power Users

One of the challenges of building an analytical platform is that there isn't one universal analyst. Some users want to interact primarily through a graphical interface. They want to click, search, navigate relationships, and complete workflows without writing queries.

Others want more control. They want to write Storm, build macros, automate processes, create Power-Ups, interact with APIs, and push the platform in directions its developers may never have anticipated. Synapse has increasingly been designed to accommodate that entire spectrum.

Features such as the Threat Intel workflow make complicated relationships accessible through purpose-built interfaces, while the Storm language and its broader APIs give advanced users the ability to build much more sophisticated workflows. The objective isn't to turn every analyst into a developer. It's to make the power of the platform accessible at whatever technical level makes sense for the person using it.

What Ten Years of Collaboration Built

When asked which collaborative accomplishment they're most proud of, the group doesn't arrive at a single answer. Epiphyte points to the breadth of the Storm library, type system, and APIs that allow advanced users to interact with everything from HTTPS endpoints to files and structured data. Redox highlights the Threat Intel workflow, which combines new engineering capabilities with analyst input around how complex intelligence relationships should actually be represented to users.

Thesilence points to the Synapse data model itself: the foundation that determines whether analysts can accurately represent reality, query their information, and understand the relationships within it. And savage brings those answers together - the larger achievement isn't necessarily one individual feature. It's building a platform capable of serving the entire spectrum of users from someone who wants to look up a threat group and explore what it has targeted, to a power user writing sophisticated Storm and extending the platform themselves. That range is possible because analysts and engineers built their pieces together instead of individually.