Introduction

A long time ago, back in 2017 (seriously, how was that 8 years ago), I wrote a post about my security talk development process. I stand by it, but I think it’s time to revisit it. I’ve learned a lot since then, and I think the process has changed a bit. I was lucky enough to be asked to give a talk at SANS Emerging Threats Summit in 2025, and I thought it would be a good opportunity to share my current process. I’ll go into some of the details of the talk itself, but I will share my process for developing it. I think it’s a good example of how I approach talk development in general, and I hope it’s helpful to you.

TLDR: Building Better Security Presentations

Cartoon of me Speaking What ChatGPT thinks I look liked like speaking at SANS Emerging Threats Summit 2025

Let us start with the major takeaways my original post, which I believe are still valid today:

TLDR: Your images are too small, your fonts are too small, you have too many words, your colors don’t have enough contrast.

But the full process is a bit more involved. Here’s the process I outlined in my original post:

  • Write the abstract
  • Write the outline
  • Outline slides
  • Pick a theme
    • Pick a color palette
    • Pick a font
  • Build the slides
  • Practice the talk
  • Give the talk
  • Get ongoing value

All still fair. But really this outline can be thought of differently:

  • Process: Write the abstract
  • Process: Write the outline
  • Process: Outline slides
  • Tool: Pick a theme
    • Tool: Pick a color palette
    • Tool: Pick a font
  • Process & Tool: Build the slides
  • Process: Practice the talk
  • Action: Give the talk
  • Process: Get ongoing value

So this is largely the same outline, but with some categorization. One of the things I’ve changed in my process is trying to use tools to help me with the process. The primary challenges of good presentations I laid out are almost always formatting: stuff (images & text) is too small and colors are non-meaningful and hard to parse. These combine to make it hard to read and understand the slides. So I try to use tools to help me with that.

So whats the problem? Well getting these formatting pieces right is hard and leads to endless tweaking. I want to spend my time on the content, not the formatting, something that became even more clear to me teaching at Utah State University. I used to think I pumped out a lot of presentations, but when you’re teaching two classes, you’re creating/updating two to four presentations a week, and often doing outside presentations as well. There isn’t enough time to finesse an image here or color there.

So how do I get around this: tools. While in my original post I outlined a handful of different presentation tools, and treated them all as viable depending on your process I’ve shifted entirely: Markdown or bust. At the point you need to generate multiple presentations a week, you need to be able to generate them quickly. And the best way to do that is to use a tool that allows you to write in Markdown.

At that point where the content is all Markdown you can separate the presentation from the content. You can use a variety of different tools to generate the presentation, and you can use a variety of different tools to manage the content. You can also make the tools enforce all of our goals around colors, sizes, images, and layouts.

Markdown for Presentations

Markdown Cheetsheet

Markdown is a lightweight markup language that allows you to write in plain text. It’s easy to read and write, and has a variety of different formatting options. Some are built in, like # for headers and * for lists, but some are not, like --- for slide breaks. Markdown can be managed as code, so you can use version control to track changes and collaborate with others, or even us linters and code checkers to ensure quality. I can even use my IDE to write my presentations, and use tools like Markdownlint to ensure that my Markdown is valid. This is a huge win for me, as I can use the same tools I use for writing code to write my presentations. Many code based presentation tools even have VSCode extensions.

In that vain something else has changed since 2017: the emergence of Large language models (LLMs). LLMs are great at generating text, but is it high value text? Well, it depends. I think they are great at generating outlines and helping with the process of writing. I leverage them a lot for outlining and brainstorming. I also use them to help with the writing process, but I try to avoid using them for the actual content.

Another great thing about Markdown though is how easy it is to move between tools. There are a variety of different tools that can convert Markdown to presentations. I started by using DeckSet, a MacOS/iOS based tool that allows you to write in Markdown and then generate a presentation. It has a variety of different themes and options, but it’s not open source and is MacOS/iOS only. I also didn’t love the way it manages presentations as bundles, as in my classes I used the same images in multiple presentations, which works great on MacOS, but not so well on iOS. I’ve since moved to Marp, which is an open source tool that allows you to write in Markdown and then generate a presentation. It also has a variety of different themes and options, and, unlike DeckSet, it’s cross platform. It also has a VSCode extension. Marp has a lot of moving pieces though, especially as I’ve integrated tools like Mermaid.js to generate diagrams and flowcharts, added my own themes, etc.

For my SANS Emerging Threats Summit talk, I decided I wanted to try something new, so I used Slidev, which is yet another Markdown presentation tool that allows you to write in Markdown and then generate a presentation. Slidev seems to have a more unified package, active development community, plugin infrastructure, and has a lot of great features like screen recording. It also has a VSCode extension to speed up Slidev creation.

Writing the Abstract

Most of the process pieces stay the same. I wrote the abstract months ago, and didn’t think to capture the process. Here’s the abstract that got accepted.

LLM SATs FTW

AI has been set to revolutionize every aspect of cyber security in the next 6 months… for the last 3 years. Cyber Threat Intelligence is supposed to be the exact kind of high intensity knowledge work where LLMs were supposed to make human analysts obsolete. We will look at where AI systems can and can’t support analysts, rather than replace them, by making the exact techniques analysts should do, but often can’t, possible.

BSidesSF2024

My original idea is based on expanding work I did in BSidesSF talk in 2024 where I tried replacing my intern Chandler with LLM based tools to mixed result. I wanted this talk to continue that story, and it’s focus on experimentation rather than assumption, but going into more rigorous human driven analysis approaches like Structured Analytic Techniques.

Outline Generation

My speaking slot was 15 mins, so I need to keep it short and sweet. I wanted to focus on the following questions, derived from the abstract:

QuestionAnswer
What is the problem?SATs are hard, take too long, and are often not done. Maybe LLMs can help?
What are the SATs I want to use?Three is a nice round number. Everyone does ACH in teaching, but rarely ever do ACH, so lets do that. I want an anticipatory intelligence aspect, so lets also do Lastly lets do an Red Teaming SAT, as I think this is a great way to show how LLMs can help with the human side of things, especially for small teams.
How can we do these SATs with LLMs?I want to show how LLMs can help with the human side of things, especially for small teams.
What are the results & limitations?I want to show how LLMs can help with the human side of things, especially for small teams.
What are the next steps?I’m not sure I know yet…

Thats a lot to answer in 15 minutes, but we’re aiming for high speed, low drag, and density. Here’s the outline:

# LLM SATs FTW

- Introduction (Short)
    - Who Am I?
    - What is the threat?
        - Bias
        - Budget
        - Boredom
    - What are SATs?
    - Why are SATs hard?
    - Why are SATs important?
- Body
    - SAT #1: Key Assumptions Check
        - What is Key Assumptions Check?
        - Why is Key Assumptions Check hard?
        - How can LLMs help with Key Assumptions Check?
        - Demonstration
        - Results & Limitations
    - SAT #2: ACH
        - What is ACH?
        - Why is ACH hard?
        - How can LLMs help with ACH?
        - Demonstration
        - Results & Limitations
- Conclustion (Short)
    - Results & Limitations
    - Jevons Paradox
    - AI (Artificial Intelligence) -> IA (Intelligence Augmentation)
    - Takeaways
    - Share Resources
    - Follow up contact

Now I want to be real: I didn’t write all this. Well… I did, but GitHub Copilot made a lot of inline suggestions, some of which I just happily hit tab to accept, often with follow up modification, but I think it’s a good scaffolding to start with.

Building the Content

The point of this talk is exploring weather or not LLMs can help with SATs, so I needed to build some tools to help with that. If you want details on the tools I built, check out my paired post with this one LLM SATs FTW. I built three tools, one for each SAT, and used them to generate the content for the talk. The tools are all built in Python, and use Streamlit to build a simple web app that allows you to enter a topic and then generates a series of questions about that topic. calling out to OpenAI.

Outputs

Along the way I took screenshots of the tools I built, and the basics of how they work, which will be the basis of a lot of my slides. I also took screenshots of the LLM outputs, and the CSVs generated by the tools. These plus my outlines were enough to build my slides. I also took a lot of notes along the way, and I think I can use those to help with the slides as well.

Building the Slides

Now for Slidev. What I love about Markdown based slide tools is that they are all very similar and most work best with my writing process. I wrote a simple outline, then added the content detail, followed by the adding the tool specific directives and syntax for formatting. Given this was my first time with Slidev my goal is not to try to do too much, avoid all the bells and whistles, and just get the content out. I can always go back and use the cool features after the fact. Slidev initially creates it’s own example presentation, so I copied that to an example file, hacked out most of the base content in slides.md and then started adding my content.

Now I called out the steps “Pick a color palette” and “Pick a font”. I could have… but I didn’t. In this case I just used the default theme, and the default font. I think this is a great way to get started, and I can always go back and change it later, part of the benefits of using a code driven approach.

I removed all but the first slide, then used my outline to build basic slides, just text, with a focus on sections and headers. The goal was to give a structure, a framework, a skeleton. Now there two directions to go at this point: one could go back and add the content, or start adding the formatting, images, and other eye candy. I went with content first, as I think it’s easier to add the formatting and images once I have the content in place. This is key as content may change, and it should drive the formatting, otherwise you end up in a loop of trying to get the formatting right, and then changing the content, and then trying to get the formatting right again. I also want to make sure that the content is clear and easy to read, so I want to focus on that first. Along the way I realized a few parts of my experiments could be improved, and I made some changes to the tools, reran the experiments, and took new screenshots. I also added some notes to the slides about the experiments, and how they worked. I think this is a great way to show how LLMs can help with the human side of things, especially for small teams.

The results I think are pretty solid. See for yourself:

transition theme title description author lang layout
none
default
LLM SATs FTW
Augmenting Analyst Decision Making with AI driven Structured Analytic Techniques
Scott J Roberts
en
intro

LLM SATs FTW

Augmenting Analyst Decision Making with AI driven Structured Analytic Techniques

Scott J Roberts - SANS Emerging Threat Summit 2025

Scott J Roberts

  • Instructor of Cyber Security
    @ Utah State University
  • Founder @ Taurus.blue
  • Author of Intelligence Driven Incident Response with Rebeakh Brown
  • Former SANS 578 Instructor

layout: section

Threat, Problems, & Solutions


level: 2 layout: fact

Threat:
Cognitive Bias

A systematic deviation from the truth based on System 1 thinking.


level: 2 layout: quote

"Structured analysis is a mechanism by which internal thought processes are externalized in a systemic and transparent manner so that they can be shared, built on, and easily critiqued by others."

~ Structured Analytic Techniques for Intelligence Analysis by Heuer & Pherson


The Problem

SATs are hard to learn, hard to use, hard to teach, work best in teams, and take too long to execute effectively.


level: 2 layout: fact

Analysts + SATs + LLMs == Profit??


layout: section

Experiments


SAT: Starbursting

"Starbursting is a brainstorming technique that focuses on generating questions rather than eliciting ideas or answers. It uses the six questions commonly asked by journalists: Who? What? How? When? Where? and Why?"

~ Structured Analytic Techniques for Intelligence Analysis by Heuer & Pherson


title: Starbursting

Starbursting

  • Built a Streamlit app to run the Starbursting SAT
  • Zero shot based on the SAT
    • Given a scenario, generate questions for who, what, when, where, why, and how
  • Output a JSON file with the results for human review
  • Test Case: A ransomware attack on a hospital

title: Starbursting Screenshot 1 layout: image image: images/starburst_1.png


title: Starbursting Screenshot 2 layout: image image: images/starburst_2.png


title: Starbursting JSON level: 2

{
  "topic": "A ransomware attack on a hospital",
  "answer_who": [
    "Who carried out the ransomware attack on the hospital?",
    "Who was affected by the ransomware attack on the hospital?",
    "Who responded to the ransomware attack on the hospital?"
  ],
  "answer_what": [
    "What was the impact of the ransomware attack on the hospital?",
    "What measures were taken to mitigate the ransomware attack on the hospital?",
    "What was the ransom demand in the ransomware attack on the hospital?"
  ],
  "answer_when": [
    "When did the ransomware attack on the hospital occur?",
    "When was the ransomware attack on the hospital discovered?",
    "When was the ransomware attack on the hospital resolved?"
  ],
  "answer_where": [
    "Where did the ransomware attack on the hospital originate from?",
    "Where were the hospital's systems affected by the ransomware attack?",
    "Where was the response to the ransomware attack coordinated from?"
  ],
  "answer_why": [
    "Why was the hospital targeted in the ransomware attack?",
    "Why was the ransomware attack on the hospital successful?",
    "Why did the ransomware attack on the hospital cause the damage it did?"
  ]
}

level: 2

Starbursting Visual Output

---
config:
  theme: neo
---
mindmap
  root((A ransomware attack on a hospital))
    Who
      Who carried out the ransomware attack on the hospital?
      Who was affected by the ransomware attack on the hospital?
      Who responded to the ransomware attack on the hospital?
    What
      What was the impact of the ransomware attack on the hospital?
      What measures were taken to mitigate the ransomware attack on the hospital?
      What was the ransom demand in the ransomware attack on the hospital?
    When
      When did the ransomware attack on the hospital occur?
      When was the ransomware attack on the hospital discovered?
      When was the ransomware attack on the hospital resolved?
    Where
      Where did the ransomware attack on the hospital originate from?
      Where were the hospital's systems affected by the ransomware attack?
      Where was the response to the ransomware attack coordinated from?
    Why
      Why was the hospital targeted in the ransomware attack?
      Why was the ransomware attack on the hospital successful?
      Why did the ransomware attack on the hospital cause the damage it did?
Loading

SAT: Analysis of Competing Hypotheses (ACH)

"Analysis of Competing Hypotheses (ACH) is an analytic process that identifies a complete set of alternative hypotheses, systematically evaluates data that are consistent or inconsistent with each hypothesis, and proceeds by rejecting hypotheses rather than trying to confirm what appears to be the most likely hypotheses."

~ Structured Analytic Techniques for Intelligence Analysis by Heuer & Pherson


title: ACH

Analysis of Competing Hypotheses

  • Built a Streamlit app to run the ACH SAT
  • Multi stage process based on the SAT
    • Accepts a complex question
    • First API Call: Generate a list of hypotheses
    • Second Set of API Calls: Generate a list of evidence for/against each hypothesis
    • Third Set of API Calls: Score each hypothesis based on the evidence
  • Output a CSV file with the results for human review
  • Test Case: Who was behind the XZ backdoor?

title: ACH Screenshot 1 layout: image image: images/ach_1.png level: 2


title: ACH Screenshot 2 layout: image image: images/ach_2.png level: 2


title: ACH Screenshot 3 layout: image image: images/ach_3.png level: 2


title: ACH Output level: 2

ACH Matrix


SAT #3 - Key Assumptions Check


SAT: Key Assumptions Check

"The Key Assumptions Check is a systematic effort to make explit and question the assumptions (the mental model) that guide an analysts interpretation of evidence and reasoning about any particular problem."

~ Structured Analytic Techniques for Intelligence Analysis by Heuer & Pherson


layout: image-right image: images/strider.png

Key Assumptions Check

  • Built a Streamlit app to run the Key Assumptions Check SAT
    • Accepts a PDF file, extracts text
    • Zero-shot classification of the text
    • Generates a list of key assumptions
  • Test Case: Strider Technologies - Inside the Shadow Network
    • The report is about North Korean IT workers and their involvement in cybercrime

title: Key Assumptions Check Screenshot 1 layout: image image: images/kac_1.png


title: Key Assumptions Check Screenshot 2 layout: image image: images/kac_2.png


Key Assumptions Check

  • The document assumes that IT workers are involved in the manipulation of cryptocurrency markets, including the use of malware to mine cryptocurrencies.
  • The document assumes that North Korean IT workers are dispatched abroad to countries like the PRC, Russia, Southeast Asia, Africa, and the Middle East.
  • The document assumes that North Korean IT workers are involved in cybercrime activities such as hacking, ransomware deployment, and intellectual property theft.
  • The document assumes that North Korean IT workers are using false identities and front companies to infiltrate Western businesses.
  • The document assumes that PRC-based entities are involved in shipping equipment for DPRK remote workers.
  • The document assumes that PRC-based front companies are facilitating the global operations of fraudulent North Korean IT workers.

[...]


level: 2 layout: fact

Results & Limitations

Well it depends...


layout: section

Conclusion


layout: two-cols-header level: 2

Jevon's Paradox

::left::

"The Jevons Paradox is when making something work better actually leads to using more of it, not less."

~PhilosophyTerms.com: Jevons Paradox

::right::

---
id: 38a3f8d5-4955-4258-9702-0cb303ee4a18
---
flowchart TD
    A[Technological Advancement] --> B[Increased Efficiency]
    B --> C[Cost or Price Reduction]
    C --> D[Increased Consumption]
    D --> E[Economic Growth]
    E --> B[Further Demand for Resources]
Loading

level: 2 layout: fact

AI → IA

Artificial Intelligence to Intelligence Augmentation


Take Aways

  • LLMs are not a replacement for Analysts
  • Let computers do computer things, let humans do human things, and figure out they work together
  • Experimentation is always better than theory
  • An AI system doesn't have to be better than a human, just better than the best available human


Contact


layout: end

Thank You!!!

view raw slides.md hosted with ❤ by GitHub

Practice Giving the Talk

One of the things I like about Slidev is the extra features, including the ability to record a presentation in browser. I walked through the presentation, and recorded it. I think this is a great way to practice the presentation, and get a feel for the timing. A bit over on time… as a result I went back and cut a few slides, and added some notes to the slides about the experiments, and how they worked.

After all the practices, edits, and tweaks, here’s the final version of the slides I submitted:

I think I’m ready for show time.

Give the Talk

Well… I gave the talk, and it went well. I think the content was solid, and the slides were clear and easy to read. I got a lot of good feedback, and I think the audience really enjoyed it. I also got a lot of good questions, which is always a good sign.

SANS Emerging Threats Summit 2025 One of the best parts of any SANS Event: the visual recap!

There were some challenges. Like many organizations, and especially in an online venue, SANS really wants you to submit your slides ahead of time. I get it… it’s a trade-off. In a positive sense it avoids flip-flopping around, getting to the right screen, sharing the right window, etc. But it also means you can’t make last minute changes, and you can’t use the tools to help you with the presentation. I think this is a good trade off, but it’s something to be aware of. The biggest issue is no speaker notes. I love speaker notes, and I use them a lot. I think they are a great way to keep track of what you want to say, and how you want to say it. I also think they are a great way to keep track of the timing of the presentation. In a positive sense it means you need to know your content cold, but it also means you need to know your content cold. In the case of using Slidev it also meant certain features weren’t available, since, like most browser based tools, Slidev generates a .pptx by essentially taking a screenshot of the slides. This is a great way to generate a presentation, but it also means you can’t use certain features, like certain animations and transitions. Largely that’s for the best, as they are often overused, but it does take a tool out of your presenting tools box.

Get Ongoing Value

Well… that’s what this blog post is really about! I love the 37 Signals concept of “sell your byproducts” and I think this is a great way to do that. My process hasn’t changed so much as evolved, but I still think it might help others to see it.

I also posted my slides to SpeakerDeck so you can check them out there (thats the embed from above). I also posted the code for the tools I built to GitHub - llm-sats-ftw-code and I’m even sharing my slides, as build in Markdown with Slidev, on GitHub - talk-llm-sats-ftw. I’m also planning to post the video when it’s shared by SANS, and posting everything to LinkedIn.

Conclusion

Overall I couldn’t be more pleased about how this talk came together, both in terms of content and process. I think the process I outlined is a great way to approach talk development, and I hope it’s helpful to you. I also think the tools I used are a great way to approach talk development, and I hope they are helpful to you as well. Markdown makes it easy to write, easy to integrate other tools, easy to save content, and easy to keep content and presentation separate. I cannot recommend Slidev (or Marp, or Deckset) enough. If you have any questions or comments, please feel free to reach out!