What is Open-Source Intelligence (OSINT)?
What is OSINT?
In “Breaking the Kill Chain: A Defensive Approach,” we took a look at the seven steps an attacker generally goes through in completing an attack. The seven steps highlight the major milestones and techniques an attacker goes through on their way to their end objective. But arguably, no step in the Cybersecurity Kill Chain is more important than Reconnaissance. The information gathered during this phase is the building block on which the rest of the kill chain is based.
OpenSource Intelligence, or OSINT for short, is the practice of collecting publicly available information using a variety of different sources. For an attacker that is beginning to plan out their attack, using OSINT tools and techniques is the first step in their journey. In this blog, we’re going to cover the OSINT framework, methodology, and tools we could use during this portion of the attack. We’ll then wrap up by talking about how blue teams should use this information to protect their public information from being in the wrong hands.
Passive Info Gathering Using OSINT
During the reconnaissance stage of the kill chain, we’re trying to obtain information that will guide the kind of attack we later use. OSINT, or Opensource Intelligence, provides a framework of tools and techniques we can use to gather information about our target using publicly available resources. OSINT is really about the practice of using public information as reconnaissance on our target. In our case, we’re interested in anything that can be used for later attacks, such as information on the network equipment, employee email address, or social media pages.
OSINT Step
The OSINT process starts with something you know about the target. Something as simple as a company name can be the starting point from which we work to acquire something we could use.
We then define what kind of information we’re after. If we’re after user credentials, for example, we know we need to acquire an email address and possibly social media accounts before we can send a targeted spear phishing campaign.
The 3rd step is where we use the tools we’ll discuss here momentarily to collect information about the target. As we’ll see next, different tools have different purposes, so knowing how to obtain the information you’re after is key to your investigation.
Next, we analyze the data we collected and, in some cases, use what we found as the starting point for further analysis.
For this blog, we’re really going to be focusing on the tools and resources that we can use to collect and analyze public data, particularly with understanding what tools can be used for different pieces of information. And perhaps the best place to start is with the OG of OpenSource Intelligence, and that’s Maltego.
Maltego
Maltego is a powerful data mining tool that can search thousands of online data sources to find connections between pieces of information. This is accomplished using a series of “transform” steps that essentially automate the lookup process while also providing you with a visual layout of the information as it is learned. For example, we can start off by typing in the domain name of an organization and right-clicking to select transform or task we would like to run and then move on to the next transform or task on that piece of information. For example, we can use a domain name to find DNS records that point to real public servers. We can then transform those servers to IP addresses and easily find other blocks of IP space that belong to that owner.
Part of Maltego’s power is in the visualization and connections to pieces of information as it is obtained. This is extremely useful as we move to later steps of the kill chains in plotting out areas we may want to focus our attack on. While the learning curve is a bit steep, Maltego is extremely powerful. It also provides 3rd party plugins support to query other data feeds like Shodan as a transform action.
theHarvester
theHarvester is a slightly different kind of tool because it focuses on popular OSINT search engines like Google, LinkedIn, and Shodan as the main sources of data. However, this insanely easy yet powerful tool can be quite useful in finding valuable data about our target.
Spiderfoot
Another powerful but lesser-known OSINT tool is Spiderfoot which can consolidate hundreds of data feeds into a single search. Unlike Maltego, where you have to specify the specific action you want to run on a given target, Spiderfoot is like Google search that queries nearly all the publically available OSINT sources available.
Using Spiderfoot is as easy as typing what we know about the target, such as a username or company website, and selecting the type of scan we want to run. With hundreds of modules and API connections to various OSINT resources, they also make it easy by grouping them based on the kind of information you’re looking to obtain. If you select “Passive” Scan, it will not send any direct queries from your machine by removing modules that attempt to make direct connections.
While I found Spiderfoot to be much more intuitive than Maltego, it does not visualize the data as well. However, the results are categorized in a useful way that makes analyzing it relatively simple.
Misc Tools
Maltego and Spiderfoot are fantastic OSINT tools that should be leveraged during our investigation, but they’re certainly not all-encompassing. Different tools may be needed for different things, so here’s a brief overview of some of the other tools at our disposal:
- Babel X is a multilingual search tool that can span across many OSINT resources in different languages. This is particularly useful when researching targets that communicate in other languages
- Recon-ng is more of a development tool based on Python that allows you to develop the kind of searches you want to use based on modules. The benefit here for developers is the ability to automate the OSINT process into your applications by leveraging the Recon-ng framework.
- Metagoofil is meant to extract metadata from public documents. This is extremely useful for research things like business owner information or potentially sensitive information that is stored in cleartext.
No OSINT research is complete without mentioning traditional OSINT sources like Google Hacking, SHODAN, WayBackMachine, and NetCraft. The reality is that there is simply not enough time to cover these in any great detail, but they are worthy of your OSINT research toolbox.
OSINT Framework
I’d also encourage you to check out and bookmark OSINTframework.com, as this provides an excellent visual and updated list of links that can come in handy during your investigation.
While OSINTFramework.com provides a great reference point to some of the tools at our disposal, it’s really just the tip of the iceberg in terms of what OSINT is all about and what tools are available to us. Ideally, we’d want to not just query all these sources for information about our target but visually connect that data in a way that’s useful. To accomplish this, we are going to look at some of the more powerful OSINT tools at our disposal.
Wrap up – defense
There’s no question about the value good OSINT research can have for attackers and Red Teams as they prepare their journey through the kill chain. From a blue team or defensive perspective, it’s equally as important that we understand what information is out there about not just our organization but our employees as well.
If you want to be proactive about defending against OSINT research in your organization, start by implementing regular scans on your organization and its employees. Make time to determine whether the information you found must be public or if it can be removed from public access. If you want to take it a step further, you can sign up for Dark web monitoring services that will notify you when your organization’s information has made its way to dark websites and forums, which is a breeding ground for would-be attackers.
That’s why regular OSINT scans should be performed to understand the data that is potentially available to would-be attackers, as well as understanding how that data can be used to harm the organization and its users.
