SkipTracer | an OSINT Scraping Framework | Basic Python Webscraping Of PII Paywall

Tools & Scripts
, , ,
1

image
image728×380 79 KB

Introduction

Skiptracer is an OSINT scraping framework that utizes basic python webscraping of PII paywall sites to compile information (passively) on a target. This is especially useful if you are operating on a very small budget. Skiptracer allows you to search by specific input including:

  • Email
  • Name
  • Phone
  • Screen Name
  • License Plate
  • Domain

It’s primarily US-focused, but you can use tools like LittleBrother for possible access to the EU domain.

For email search, Skiptracer checks LinkedIn, HaveIBeenPwned, MySpace, and AdvancedBackgroundChecks. For name search, Skiptracer leverages Truth Finder, True People, and AdvancedBackgroundChecks. For phone numbers, Skiptracer searches TruePeopleSearch, WhoCalld, 411, and AdvancedBackgroundChecks. For screen names, SkipTracer checks Knowem, NameChk, and Tinder. For license plates, it uses Plate Search. For domains, is uses Subdomain Search.

While there are tools that do many of these tasks better, in my opinion, there are no tools I’m aware of that do all these tasks in one place. It’s sort of a Swiss army knife of people search tools that’s an excellent place to start your investigation.

People Search

People searches function as one of the most common OSINT investigations. Whether its private investigators, penetration testers, or journalists, gathering information on people is fundamental for many operations. Changes in social media policies and API access as well as public policy changes has changed the game for many OSINT investigators in this domain; however, it’s important to note that changes that make OSINT collection more difficult actually increases the value of the OSINT skill set. Tools like Skiptracer allow you to start off with more information than the average citizen could gather after exhausting all of their known resources. Considering this tool serves as a starting point for investigations, the value added after leveraging other tools can be priceless.

Skiptracer Setup

One thing that I don’t like about Skiptracer is that it’s written in Python 2.x. However, I understand why it’s written in an older version of Python and recognize that many other tools are written that way too. That being said, Skiptracer is very easy to setup. It only requires Python 2.x, and Git. You don’t need to input any API keys, which is relieving. Here’s how you set it up in terminal.

git clone https://github.com/xillwillx/skiptracer.git skiptracer
cd skiptracer
pip install -r requirements.txt
python skiptracer.py

Now, you may run into some issues when installing requirements with pip. You may get an error code stating something about not having permissions. This is common for installing packages with pip. In the event you see this, try this command instead.

pip install --user -r requirements.txt

This will give you permissions to install the required packages to that user account instead of to root. You may be tempted to install packages from root using sudo, but that’s not recommended. Once you get all the packages installed, go ahead and run the script again and you should see something like this:

image

Use Cases

Learning how to pivot with the information you have in an OSINT investigation is a primary skill set you have to learn if you hope to produce a comprehensive report of findings. Skiptracer allows you to endlessly pivot from one data point to the next.

For example, let’s say you have an email address you’re investigating. You want to find out who owns it and any additional information you can find about the user. You run the email address through HaveIBeenPwned using Skiptracer and discover it was part of a Disqus data breach. This reveals that the owner of that email address may have content published on Disqus, a popular engine for hosting comments within articles online. Armed with this information, you can pivot to find the information required to locate comments they’ve made through Disqus, if any at all.

OSINT Insight

Skiptracer is a great tool to start an investigation, but it lacks a few features that other tools make up for when used in combination. I don’t criticize the lack of social media searches or IP address searches because the former is difficult to do and the latter doesn’t exactly fit the mold of the product. I do think that for usernames you should use a tool like Sherlock of Socialscan as Namechk and Knowem have been proven to be unreliable and should only be used if you don’t have access to Python scripts. Because the purpose of this tool is to get the most amount of information possible without spending coin, I commend it on its efforts. I do think that expanding the tool to encompass API keys that a user may need to input, such as FullContact, would add value to the product. Here’s my recommended workflow for a basic OSINT investigation using Skiptracer.

= pivot

Initial request > Skiptracer (email/name) > Sherlock/Socialscan (username) > ThatsThem (phone number/address) > Manual search to match located information to social media profiles (Google Dorks) > Twint (Twitter) > InstaLoader (Instagram) > Searchbook (Facebook) > Initial report

ENJOY! :+1:

11 Likes