ACH Analysis of a Trump Campaign Compromise

December 12, 2016
Intelligence

ASIDE: This post gets political. People may agree or disagree based on their own experience or personal belief. I accept that. I’m attempting to use evidence and analytical rigor to reach my conclusions while averting my own bias. If you think I missed the mark on those aspects (evidence or rigor) feel free to reach out to me. If you just disagree with my conclusions then I’d love to see a blog post exploring your own evidence and process.

The news this past week has been filled with statements about claims and counter claims of whether or not Russia was involved in meddling in the 2016 Presidential Election. Intelligence agencies, news outlets, & security vendors have stated that Russia compromised the Democratic National Committee, Hillary for America campaign Chairman John Podesta, Democratic Congressional Campaign Committee, and others related to the campaign. During the debates Donald Trump questioned the validity of this, leading to the 400 lbs hacker line (and resulting Twitter accounts) and he and his transition team have repeatedly questioned/refuted all such claims, even suggesting that the entire intelligence community might be wrong.

What’s been more interesting to me though has been questions about whether or not the Republican National Committee and Donald J. Trump for President, Inc were victims of similar attacks (For brevity we’ll just refer to them all collectively as the Trump Campaign). Many of us working in intrusion detection and incident response have had conversations with peers from other companies in the same vertical describing identical attacks. In fact this is one of the major strengths of computer network operations: hacking represents an asymmetric threat, having relatively low variable costs (the effort to compromise a target) vs fixed costs (the development of exploits and tools). I will say I for one assumed that the same attacks directed at the Democratic institutions were also directed at the Republican institutions.

The Republican leadership has said little about any of this until this weekend:

NBC News - Priebus on Russia’s Influence: ‘I Don’t Know Whether It’s True’ Despite CIA report, outgoing RNC Chairman Reince Priebus says that he isn’t sure Russia tried to influence the election.

I for one was hugely surprised to hear not simply the assertion that the Trump Campaign hadn’t been hacked (an absence of evidence not being the same as evidence of absence), but the unabashed confidence. I realized that this surprise (even incredulousness) was based on my own experience and assumptions, biases I have, instead of on any analytic rigor. Sunday morning with coffee is obviously the best time to apply intelligence rigor to a problem, so here goes.

One powerful analytic model for problems of conflicting information and rampant bias is Analysis of Competing Hypotheses. A system of listing important information and scoring it to weed out bias and develop relativistic scoring of likely possibilities. ACH doesn’t give the answer but helps analysts understand what is most likely. Descriptions of each step are pulled from the CIA’s Center for the Study of Intelligence article linked above.

Hypothesis

Identify the possible hypotheses to be considered. Use a group of analysts with different perspectives to brainstorm the possibilities.

I can come up with five serious hypotheses specifically with regard to Russian intelligence gathering activity:

These don’t cover 100% of all possibilities (like if the adversary has a time machine and is getting intelligence sent from the future) but covers the major possibilities.

Evidence

Make a list of significant evidence and arguments for and against each hypothesis.

Next we collect evidence for and against our hypotheses. Evidence is based on publicly disclosed information and some are based on experience from 10+ years as a defender and incident handler. Some of it is circumstantial, but it’s the best we have to go off of.

The Trump Campaign was compromised and they are unaware.

The Trump Campaign was compromised and they are aware but want to withhold it.

The Trump Campaign was not compromised because their security was so good it blocked all attempts.

The Trump Campaign was not compromised because no one attempted.

The Trump Campaign was not compromised because the adversary was collecting via non-technical means (IE HUMINT or collaboration).

Matrix

Prepare a matrix with hypotheses across the top and evidence down the side. Analyze the “diagnosticity” of the evidence and arguments–that is, identify which items are most helpful in judging the relative likelihood of the hypotheses.

Taking the theories and evidence above everything goes onto a matrix without any editing or scoring. Fields have been abbreviated or coloquilized layout.

Looking at the matrix a handful of pieces of evidence stand out as diagnostically significant:

Refine Matrix

Refine the matrix. Reconsider the hypotheses and delete evidence and arguments that have no diagnostic value.

Here we add and remove a few pieces of evidence.

I didn’t change my hypotheses. I still feel strong about all of them.

Tentative Conclusion

Draw tentative conclusions about the relative likelihood of each hypothesis. Proceed by trying to disprove the hypotheses rather than prove them.

Now the process moves on to scoring. The scale is +2 to -2. The +2 should indicate a very strong correlation, -2 means a strong contradiction. A score of 0 indicates evidence neither confirms or denies the hypothesis.

This isn’t the end yet! Now it moves on to the adjustment of bias.

Adjust for Bias

Analyze how sensitive your conclusion is to a few critical items of evidence. Consider the consequences for your analysis if that evidence were wrong, misleading, or subject to a different interpretation.

This one is tough and requires considerable self reflection.

What about consequences if evidence is incorrect? In some cases this could have a fairly dramatic swing. A particularly high amount of weight is placed on the evidence that the Democratic campaign institutions were compromised. Given this has been confirmed by multiple sources I’m comfortable with that. Similar issues arise with the assumption that campaign infrastructure is likely vulnerable, but this is also supported with considerable evidence. That said a change to either of these would cause a significant shift in probability, but would not ultimately change the outcome of the analysis.

Finalize & Synthesize Analysis

Report conclusions. Discuss the relative likelihood of all the hypotheses, not just the most likely one.

Now our final matrix. Again, it’s based on a number of experience based assumptions and facts we have available.

Based on this analysis I have a moderate confidence that the Trump Campaign was the victim of offensive computer network operations, but it is unclear whether or not the leadership was aware. It seems highly unlikely that the Trump Campaign had either perfect security or was not targeted at all. Non-CNO collection is difficult to determine based on the evidence we have available.

One thing I do know is that none of these items being true are good things.

I’ve wracked my brain trying to come up with more possibilities but I’m at a loss. I’ve long subscribed to the idea “Never assume malice where ignorance will suffice” but when it comes to electing our officials and leading our country neither seems acceptable to me.

Future Analysis

Identify milestones for future observation that may indicate events are taking a different course than expected.

How could this analysis improve in the future?

Whether or not we get any of these there are plenty of unanswered questions.

My Conclusion

It’s unlikely anyone will ever know the full truth and this analysis is likely missing key information, it merely provides a framework to understand what we do know. Hopefully this demonstration shows how an organization can use ACH to understand complicated issues. This may become even more important in the future as we have to accept that political realities, domestically and internationally, are a part of the computer security world.