Golang for DFIR

One of my goals for this year was getting comfortable with a new programming language. I’ve been a Python devotee for a long time and it’s almost always gets the job done, but I wanted a little bit more. There are times Python works against you:

• Dependency Nightmares: While virtualenv and a requirements.txt file work ok for developers it can often make use by non-developers or some deployment stories quite complicated.
• Speed & Scale: This is a funny one. 99% of the time Python is more than fast enough and scales widely enough. Except when it doesn’t. Chewing through a few hundred megabytes of logs? No big deal. Gigabytes or terabytes? Not so much. Hundreds of network connections? Sure. Hundreds of thousands or millions? You might need something more.
• Discipline: Python is easy. Almost too easy sometimes. It lets you cut corners, it takes care of a lot in the background. It’s a great language for getting things done but sometimes I want to learn/control/understand more. Beyond that sometimes I want software that’s more than functional, I want it to be engineered. More structure, more safety, etc.

Working for company made up of largely developers there’s always talk about new development tools and languages. Rust and Golang have been hotly discussed in these circles (along with Erlang/Elixer/Node). I was considering both until I had a discussion at ArchCon with Liam Randall who effused about how great Golang had been for his organization. Good enough for Liam is good enough for me, so I dove in.

What is Golang?

According to the Golang Language site:

Golang is an open source programming language that makes it easy to build simple, reliable, and efficient software.

Go is a strongly typed, not quite object oriented, complied language with a syntax similar to C & Python (with a few little quirks). It’s a complied language (meaning you create an actual binary instead of just running the script like in Ruby or Python). It’s garbage collected, meaning you don’t have to deal with memory management (mostly), and highly concurrent, letting you run multiple threads easily. Golang was built to be a Google Scale language, meaning it’s meant for highly distributed large scale applications.

Ready for code? Here is Hello World in Go:

Simple and easy.

Golang also has a highly developed set of tools that include things like managing dependencies (these get statically complied into your application so end users don’t have to worry about them, just you). There are tons of libraries out there, from a well developed standard library and tons of 3rd party libraries for everything from building websites to using Yara rules.

The result? You get platform specific binaries (running native, without any dependencies) that have speed, scalability, & safety along with lots useful libraries before you even start. All without having to write lower level, higher learning curve languages like C. Pretty much just what I was looking for.

It’s not all upside. Golang is strict. It has right ways and wrong ways to do things and it enforces those ideas. It is slower to develop compared to Python but in many cases the complications are wroth it.

Why use Golang for security?

I continue to think Python is the best language to start with for Security, but there are situations where you need a little bit more. For me though the first serious benefit was the compiled nature.

I was building a tool that pulled in data from various sources to use on the commandline and with Maltego. Initially these started as Python scripts specifically built for Maltego but they had a couple problems:

• While they worked well enough for me they were buggy, no tests, inconsistent, and often needed me to jump in and fix mid-use.
• The configuration was highly specific to my machine. That’s ok for me, but getting the same transforms setup on someone else’s machine would have been hard. (Yes, we could get a TDS, but for only a handful of people that seemed like overkill.)
• Lastly as useful as it was for Maltego there were times I wanted the output on the commandline instead, to use with other scripts or tools similar to ivanlei/threatbutt. Quite a change to my codebase.

The solution? Well if you haven’t guessed by now I can’t help you, but yes, I began rewriting the scripts in Golang. The result? A single binary that anyone can run whether on the commandline or with Maltego, feels far more consistent and runs on the commandline. It also has the upside of being a bit faster and easier to understand.

Resources for Learning

One fascinating thing about people is how they learn in such diverse ways. Some people like reading while others prefer video. Some prefer diving right in. I’m a fan of a mixed approach, diving into real code, then backing out to learn some theory, then having references while I build. So here’s a little bit of everything so you can build the best Golang curriculum for you!

Hands on Resources

Go is a pretty straight forward language, especially if you’ve done a language like C, Java, or Python before. As a result one of the best ways to learn it is just diving into code itself, play with it, and see how it works. Here are the easiest ways to do that:

• Your first stop should definitely be the A Tour of Go right on the Golang website. This walks through basic core structures with actual runable code, all within a web browser, letting you avoid Golang’s somewhat tedious setup before you know it’s for you.
• After you have Golang installed the next great hands on tool is Exercism.io. This project shares example problems that run live on your own machine using built in Golang tools such as go test to learn not just syntax, but the tool chain as well. Exercism goes a step beyond by having users collaborate on improving each other’s solutions. Not to mention the Exercism commandline tool is itself written in Golang.

Sites & Blogs

The web is full of resources to help you get started with Golang (almost as much as to help you catch Pokemon). Here are a few favorites:

• Golang.org: The official site of Golang run by the project maintainers this site is a tremendous resource including software, documentation, blogs, and all kinds of other resources.
• Learn X in Y Minutes: Golang I’m a big fan of a good cheatsheet. This is the best one I’ve seen for Golang. It’s the perfect syntax reminder.
• Go By Example: The next step up from a cheatsheet, Go By Example has small digestible topics perfect for those moments you want to see variations on a particular pattern.
• Komand Tech Blog: Komand is a security automation suite built using Golang. They’re taking an open approach and sharing what they learn, including their most recent post Quick security wins in Golang (Part 1).
• Awesome Go: My go to place to find out “Is there a Golang project for X?”. There’s almost always something useful.
• Dave Cheney’s Blog: If there’s a blog to start with it’s Dave’s. He has been writing about Golang since the beginning and has some great lessons. For newbies I recommend starting with Resources for new Golang programmers and Five suggestions for setting up a Golang project.

Books

• The Go Programming Language: This was where I went after the Golang Tour and I still think the best formal resource for learning Go. It’s nicely paced, doesn’t assume too much or too little, and covers everything in the Golang core you’ll need to start building.
• The Little Book of Go: As far as open source books go (yep, it’s 100% free as in beer) I think LBoG is as good as it gets. I keep it on my desktop and reference it often. If I were a student and the $30 price tag of GoPL scared me off this is a very close second.

Other Resources

• The major conference for Golang GopherCon held in Denver most summers.
• Videos for many conferences are put online by Gopher Academy (Speaking of videos GopherVids is another gem.)
• Slides for various conferences usually show up on Golang talks or github.com/golang/go GoTalks.

Projects to Look at!

There are tons of interesting Golang projects out there on GitHub, especially around security:

I also know there’s something awesome coming out soon from Ryan Huber in Go. I don’t want to spoil the surprise, but I’d keep an eye out.

Companies Using Golang

Lots of security companies are making Golang part of their stack (and thus a good language to learn if you want to work for those companies):

• CrowdStrike
• FireEye
• Komand (as previously mentioned)
• Kolide
• Demisto

And I’m sure more are looking into it. Hopefully you will be too.