The OSINT Newsletter - Issue #16

Developer tools for OSINT

Once you’ve mastered the basics of OSINT tools, tactics, and techniques and you’re ready to move onto more advanced topics, a logical next step is to learn how to investigate under the hood, so to speak, of a web page to find valuable information that either might not be present on the front end of the website or might be useful for OSINT automation. You can do this using Developer Tools.

Recently, I’ve been doing a lot of research on automating username, email, phone number, and other lookups on popular social media platforms to determine how prevalent or used a data point is relative to its online activity. I’ve been able to build these automations by understanding requests that are happening on a website and how they can be useful for open source intelligence.

Let’s take a closer look at what I mean by using a relatively new social media platform, Cohost.

What is Cohost?

Cohost is a new social media platform built from the ground up by a small team of developers and designers who like sharing things on the internet.

---

OSINT Tools

In a typical developer toolkit, there are several tabs including Elements, Console, Sources, Network, Performance, Memory, Application, etc.

In this issue, we’re going to take a closer look at the Network view.

“The Network view allows you to see and analyze the network requests that make up each individual page load within a single user's session.”

We’re going to use this to analyze Cohost and what useful information is available.

To get to Developer Tools, right-click on your browser and click “Inspect”; alternatively, you can use option + cmd + J if you’re on a Mac or shift + ctrl + J if you’re on Windows.

Developer Tools - Network Payload

The payload view in the Network tab of Developer Tools displays the data that is being sent between a client (such as a web browser) and a server in a network request.

The payload view in the Network tab of Developer Tools can be a useful tool in OSINT investigations because it allows you to see the data that is being sent between a client and a server in a network request. By inspecting the payload, you can see what information is being transmitted, such as search queries, login credentials, or other sensitive data.

In this example, you can see Cohost does a check the moment you put in a username to determine whether or not it’s already registered on Cohost or not.

The first red arrow shows that I’m on the network tab. The second shows the Payload view. The third shows what’s in the payload, in this case, the username ‘jakecreps’. The final red arrow shows which request I’m inspecting. As you can see in this example, Cohost did a check after every single letter I input.

Here’s the same example with an email address. I’ve redacted the email address for OPSEC; however, as you can see, similar to username, it checks every single letter I input against their API to determine if the email is already in use or not.

This will be very insightful in the next step.

Developer Tools - Network Response

The response view in the network tab of a web browser's Developer Tools shows the server's response to a network request made by the browser.

When a network request is made, the server responds with a status code and response headers, followed by the response body. The response view displays all of this information in a structured format.

The response view in the network tab of a web browser's Developer Tools can be useful in OSINT investigations because it allows you to examine the content and structure of a website's response to a network request.

After confirming the proper request with the payload you’re inspecting, navigate to the “Response” tab in the Network view. Here you’ll see a JSON response that indicates that the email is already taken.

You’re probably thinking, why is this useful? I can already see in the UI that the email address is already taken!

That’s a good point.

Later on in this issue, I’ll show you how you can capture this request and run it with any email address you want without having to be on Cohost at all.

But first, let’s talk about the difference between POST and GET requests and why we’re focusing on POST requests in this issue.

Developer Tools - POST Requests

POST requests are a type of network request that is sent from a client (such as a web browser) to a server to submit data for processing. In contrast to GET requests, which are used to retrieve data from a server, POST requests are used to send data to a server, typically in order to create, update or delete resources on the server.

Put in plainer terms, POST requests are when you input information on a website and receive something back, whether visible or not.

The Cohost example we used before was a POST request. We’ll focus on these moving forward.

You might be thinking, there are hundreds of requests in the Network tab! How do I find the important stuff?

Fortunately, Chrome and other browsers allow you to filter requests using search parameters. Think of it like Google Dorks for your Developer Tools. In order to remove all requests besides POST requests in your browser, use the following query:

-method:GET -method:OPTIONS -method:PUT

In Chrome, there’s a not-so-obvious search bar on the top left section of the Developer Tools. See the first red arrow for reference. After entering this query, you’ll see the only remaining requests are POST requests.

This filter should apply as you continue to navigate the website, leaving you with only the interesting requests, in this context, to analyze.

HTTP Headers - Extension

The "HTTP Headers" browser extension is a tool that allows you to view and modify the HTTP headers of a web page's network requests. It can be used to analyze and modify the headers of both request and response data.

What makes HTTP Headers awesome and why you should use it instead of the standard Developer Tools is that it allows you to switch in between tabs seamlessly, making your analysis of requests on multiple websites much more efficient.

Remember #OSINT != tools. Tools help you plan and collect data but the end result of that tool is not OSINT. You have to analyze, verify, receive feedback, refine, and produce a final, actionable product of value before it can be called intelligence.

This is the end of the free subscription to The OSINT Newsletter. To see practical applications of OSINT tactics and techniques, please consider supporting this publication with a paid subscription.

Paid subscribers get access to practical OSINT tactics and techniques I haven’t published publicly online. This issue explores practical applications for using Developer tools including how to build your own email OSINT tool with cURL and Python.

---