Limited Series Podcast

TLDR: Key Takeaways

• Investigations begin with context: Whether starting with an indicator, a question, or an observed pattern, context helps analysts determine what matters and where to focus their efforts.
• Data alone isn't enough: Tools can surface relationships and connections, but analysts must decide which signals are meaningful and which are simply noise.
• Confidence comes from evidence: The ability to show how conclusions were reached is often more important than the conclusions themselves.
• Malware is a clue, not an answer: Malware can provide valuable context, but relying too heavily on tooling alone can lead to incorrect assumptions.
• Skepticism is a core analytical skill: Strong analysts continuously challenge their own hypotheses and consider alternate explanations.

Show Notes

Every investigation starts somewhere, but not always in the same place.

In this episode of The Vertex Project's 10-Year Anniversary limited series, Kali Fencl sits down with Vertex analysts Ryann Hallback (reign), Jennifer Kolde (thesilence), and Mary Beth Lee (savage) to explore the investigative mindset that drives intelligence work. From the first moments of examining a suspicious indicator to validating conclusions and avoiding analytical pitfalls, the conversation offers a behind-the-scenes look at how experienced analysts approach their craft.

The discussion begins with a deceptively simple question: where do investigations actually start?

For some investigations, the answer is a specific indicator (an IP address, domain, file hash, or other artifact that requires further analysis). But as the analysts explain, many investigations begin with something less concrete: a pattern of behavior, an emerging trend, or a question that doesn't yet have an answer. Understanding the context surrounding an observation often proves more valuable than the observation itself.

How to Decide What's Worth Investigating

Analysts are constantly faced with decisions about where to spend their time.

Not every lead develops into a meaningful finding, and not every interesting question can be answered with the available data. Before investing significant effort into an investigation, analysts often evaluate whether sufficient information exists to support deeper analysis and whether the potential findings justify the resources required.

The conversation explores how this decision-making process applies not only to intelligence investigations but also to the workshops and challenges the Vertex team develops to help practitioners learn investigative techniques. Building effective scenarios requires balancing realism, available data, and educational value while ensuring participants can complete meaningful work within practical time constraints.

Separating Signal from Noise

One of the most difficult challenges in intelligence work is determining which relationships matter.

The analysts discuss how platforms like Synapse help investigators bring together diverse data sources and uncover connections that might otherwise remain hidden. However, simply discovering a connection does not automatically make it relevant.

Technology can surface relationships between indicators, infrastructure, identities, events, and behaviors, but determining whether those relationships represent meaningful intelligence still requires human judgment. Analysts must evaluate context, assess significance, and prioritize findings based on what is most relevant to their organization or mission.

As the team notes throughout the discussion, tools can help analysts find connections, but they cannot replace analytical reasoning.

Confidence, Evidence, and Showing Your Work

Confidence is a recurring theme throughout the episode, though perhaps not in the way many people expect.

Rather than focusing on confidence as a feeling, the analysts describe confidence as the ability to explain and defend conclusions with evidence. Effective intelligence work requires more than making an assessment: it requires demonstrating how that assessment was reached.

The conversation highlights a broader challenge within the intelligence industry: while organizations often have internal review processes, investigators rarely discuss their methodologies publicly. Reports frequently focus on conclusions rather than investigative workflows, making it difficult for others to understand how analysts arrived at their findings.

By making investigative processes more transparent, analysts can improve collaboration, increase trust in their work, and help strengthen the profession as a whole.

The Role of Malware in Investigations

Malware remains an important component of many investigations, but the analysts caution against treating it as a definitive answer.

Like any other tool, malware exists within a larger context. Some malware families are used by multiple threat actors, while legitimate software can sometimes be abused for malicious purposes. Relying too heavily on malware names or families can create false assumptions about attribution or intent.

Instead, malware should be viewed as one piece of a broader investigative picture that includes infrastructure, behaviors, operational patterns, and other supporting evidence.

The discussion also touches on a long-standing industry challenge: the tendency to name threat groups after malware families, a practice that can unintentionally imply relationships that may not actually exist.

Common Mistakes Analysts Make

When asked about mistakes they frequently see during investigations, the analysts point to a common theme: moving too quickly.

External pressure from stakeholders and internal pressure to find answers can encourage investigators to form conclusions before sufficient evidence exists. Analysts often want patterns to fit together neatly, but investigations rarely unfold so cleanly.

The team emphasizes the importance of reporting what is known, acknowledging uncertainty, and resisting the temptation to overstate confidence. Good analysts remain willing to challenge their own assumptions and reconsider their conclusions when new information emerges.

Curiosity, Skepticism, and Humor

Toward the end of the conversation, the analysts reflect on the habits that have served them best throughout their careers.

For reign, that means following investigative threads wherever they lead and remaining open to discoveries that fall outside established patterns. For thesilence and savage, it means maintaining a healthy level of skepticism and continuously searching for alternative explanations.

Together, these perspectives highlight a central truth about intelligence work: successful investigations depend on balancing curiosity with discipline.

And sometimes, they require a sense of humor.

Not every lead pans out. Not every theory survives contact with the evidence. Investigators routinely spend hours pursuing avenues that ultimately lead nowhere. The ability to laugh at dead ends, appreciate the occasional absurdity of the work, and maintain perspective can be an important safeguard against frustration and burnout.

After all, even threat actors occasionally leave behind jokes, Easter eggs, and unexpected moments of levity.