Limited Series Podcast

TLDR: Key Takeaways

• Great intelligence starts with challenging your own assumptions: Confirmation bias is a natural part of human thinking, but effective analysts actively test their hypotheses instead of looking for evidence that simply supports them.
• Not every new threat needs to be connected to an existing one: Similar infrastructure or overlapping indicators don't automatically point to the same threat actor. Strong analysis follows the evidence rather than forcing attribution.
• Healthy analytical cultures make room for constructive disagreement: The best intelligence teams encourage analysts to question conclusions, defend their reasoning, and improve each other's work without making it personal.
• More information doesn't always lead to better intelligence: Today's cybersecurity landscape is filled with repeated reporting and "echo chamber" effects, making it more important than ever to validate information before drawing conclusions.
• Being willing to ask questions is a professional strength: Whether you're new to the field or decades into your career, curiosity, humility, and a willingness to learn are essential parts of good analytical tradecraft.
• Mentorship and community help analysts grow faster: Learning from experienced practitioners, seeking feedback, and building professional relationships can strengthen both technical skills and long-term career development.

Show Notes

No analyst is immune to cognitive bias. While technical skills improve with experience, many of the most common analytical mistakes stem from how people naturally process information rather than a lack of technical knowledge.

The team identifies confirmation bias as one of the most persistent pitfalls in cyber threat intelligence. Once analysts develop an early hypothesis, it becomes surprisingly easy to interpret every new piece of information as supporting evidence. Human brains are wired to recognize patterns, but in intelligence work, that strength can quickly become a weakness if assumptions go unchallenged.

The discussion also highlights another common tendency: trying to immediately connect new activity to a previously known threat group. While identifying relationships between campaigns is an important part of intelligence work, analysts caution against forcing connections before sufficient evidence exists. Sometimes new activity deserves to stand on its own until additional context emerges.

Strong Analysts Challenge Ideas - Including Their Own

One of the most valuable habits an analyst can develop is becoming comfortable with uncertainty.

Early in their careers, many analysts assume that everyone around them already has the answers. Reign reflects on entering the field expecting sophisticated technology, polished workflows, and experts who had everything figured out. Instead, she found that even experienced organizations often rely on imperfect tools, evolving processes, and incomplete information.

That realization changed how she approached collaboration. Rather than assuming others are always correct, analysts should feel comfortable contributing their own observations and questioning conclusions when something doesn't seem right.

This group of analysts agree that this isn't simply an individual responsibility, it's largely a cultural one. Organizations that encourage respectful disagreement produce stronger intelligence because analysts are expected to defend their reasoning rather than simply defer to expertise.

Healthy Debate Produces Better Intelligence

Constructive disagreement is a recurring theme throughout the conversation.

Analytical discussions should never be about winning an argument. Instead, they should focus on producing the most accurate assessment possible. Challenging a colleague's conclusion (or having your own challenged) isn't a personal criticism. It's an opportunity to identify gaps in reasoning before intelligence reaches decision makers.

The analysts emphasize that this mindset requires humility. Being proven wrong during an internal discussion is far preferable to publishing an incorrect assessment.

Whether teams work remotely or in person matters less than whether leadership creates an environment where questions, alternative viewpoints, and critical thinking are encouraged. When those behaviors become part of the organization's culture, analysts are far more likely to speak up when something doesn't make sense.

The Cyber Threat Intelligence Landscape Has Changed

While analytical biases haven't changed too much over the past decade, the environment surrounding analysts certainly has.

Today's intelligence teams face an overwhelming volume of reporting from vendors, researchers, and open-source communities. With so much information available, analysts must spend more time determining which reporting is genuinely independent and which simply repeats earlier findings.

The panel refers to this phenomenon as an "echo chamber." One organization publishes research, several others repeat the same indicators or conclusions, and before long the activity appears far more widespread than the underlying evidence actually supports.

This creates additional pressure for analysts to produce compelling stories quickly rather than carefully validating conclusions. The team argues that intelligence should prioritize usefulness and accuracy over sensational headlines or marketing-driven narratives.

Attribution Requires More Than Shared Indicators

Threat attribution is one of the areas where analytical shortcuts can become especially costly.

The Vertex analysts share an example involving a malware sample intentionally created for internal testing. After the sample appeared publicly, another organization quickly attributed it to a known threat actor simply because it contained infrastructure previously associated with that group.

Later, it was found the broader technical evidence didn't support the attribution. The malware's behavior, tooling, and tradecraft failed to align with the group's known tactics, techniques, and procedures.

The experience illustrates an important lesson: a single overlapping indicator rarely tells the whole story. Strong analysis requires evaluating the full body of evidence rather than assuming every connection represents a meaningful relationship.

Know When You're Building a Story Instead of Testing a Hypothesis

One of the clearest warning signs discussed during the episode is when an investigation begins to feel too perfect.

Savage compares it to collecting shiny objects - each new piece of evidence feels satisfying because it fits neatly into an existing narrative. When analysts become excited that every new observation reinforces their original conclusion, it's often time to step back and reassess.

The panel encourages analysts to continually ask themselves difficult questions:

• Am I evaluating each piece of evidence independently?

• Could this information support a different explanation?

• If I were the adversary, how might I intentionally manipulate these indicators to mislead investigators?

Those questions help prevent analysts from becoming storytellers instead of investigators.

Building Better Habits Early in Your Career

When asked what advice they would offer junior analysts, the conversation returns to a common theme: curiosity.

The team encourages new analysts to ask questions, even when they worry those questions might seem obvious. Technology evolves constantly, and no one understands every platform, protocol, or attack technique. Asking for clarification isn't a sign of weakness - it's an essential part of learning.

They also recommend seeking formal analytical training whenever possible. Technical expertise is important, but structured analytical techniques, cognitive bias awareness, and reasoning frameworks are equally valuable. Developing strong analytical tradecraft early helps analysts produce more rigorous intelligence throughout their careers.

Perhaps most importantly, analysts should remember that everyone occasionally reaches incorrect conclusions. Learning from those moments is part of becoming a better practitioner.

Community Makes Better Analysts

Not every organization has a large intelligence team.

For analysts working independently (or as part of very small teams) this panel recommends intentionally building a professional network. Thesilence stressed that mentorship doesn't always require a formal program. It can be as simple as connecting with researchers whose work you respect, participating in conferences, joining professional communities, or reaching out to experienced practitioners for advice.

The analysts also stress that community extends beyond technical questions. Having trusted peers to discuss career challenges, analytical approaches, or difficult investigations provides valuable perspective that can be difficult to find when working alone.

Strong intelligence has always been a collaborative discipline, and building those relationships benefits analysts at every stage of their careers.

Great Teams Are Built Through Culture

The conversation closes by exploring what separates effective intelligence teams from dysfunctional ones.

A cohesive team doesn't necessarily mean everyone agrees all the time. In fact, healthy disagreement is often a sign of a strong analytical culture. What matters is whether those disagreements are focused on improving the intelligence rather than winning personal arguments.

The analysts credit leadership with setting that tone. Organizations that encourage respectful debate, accountability, and continuous learning create environments where people feel comfortable challenging assumptions and improving one another's work.

Ultimately, better intelligence is produced by teams that value evidence over ego.

Final Thoughts

Every analyst will encounter cognitive biases, incomplete information, and external pressures throughout their career. Those challenges aren't signs of poor analysis. They're simply part of the profession.

The difference lies in how analysts respond. By questioning assumptions, welcoming constructive criticism, investing in analytical tradecraft, and building supportive professional communities, intelligence teams can produce work that is more accurate, more resilient, and ultimately more valuable to the people who depend on it.