The OSINT Newsletter - Issue #36
Creating a facial recognition search engine from scratch using OSINT
👋 Welcome to the 36th issue of The OSINT Newsletter. This issue contains OSINT news, community posts, tactics, techniques, and tools to help you become a better investigator.
In this issue, I will cover a pretty gray topic, face recognition. Using a few OSINT techniques, I was able to capture face encodings at scale and map them to identifiers such as email addresses, usernames, and domains with the ability later to reverse search an image of a person and if available, receive back an email or username. Stay tuned for details!
🙏 It’s 2024! Thanks for sticking with me since this publication started, restarted, and continued. At the time of this writing, this publication is just shy of 11,000 subscribers. We’re either the number 1 OSINT publication or close to it. I’m super humbled to see that.
🚨 OSINT Tool Tuesday will start rolling out this year. I’m not sure yet about the frequency; however, I might do what I did last year and start with monthly, move to biweekly, and transition to weekly as things get spun up. I’ll try to get one out sometime in January so you can see it.
🪃 If you missed the last newsletter, here’s a link to catch up.
Let’s get started. ⬇️
OSINT News
📰 The Best Alternatives to Twitter for OSINT Practitioners [2023]
The OSINT community has left Twitter. There might be a dwindling supply of researchers still active there; however, most of them are journalists covering conflict. For those looking for new communities to collaborate with OSINT enthusiasts, read this guide from OSINT Jobs.
🎩 H/T: OSINT Jobs team
📰 OSINT Tools for Diving Deep Dark Web
Apurv Singh Gautam gave a presentation at SANS about investigating the dark web. In it, they describe the tools, tactics, and techniques required for exploring the deep and dark web. Learn more about dark web networks and why you should be investigating them.
🎩 H/T: Apurv Singh Gautam
📰 The Israel-Hamas war highlights the power (and the limits) of open-source reporting
As the title suggests, this article details how both the mainstream media falls victim to open source intelligence by trusting ‘experts’ and the importance of journalists developing an OSINT skillset to avoid pontificating misinformation.
🎩 H/T: Gretel Kahn
OSINT Community
📺 Hardcore OSINT: Reversing social media mechanisms - Dmitry Danilov Soxoj 🇬🇧
This video is an absolute must-watch for readers of this newsletter. Dmitry goes in detail on how he approaches deconstructing social media platforms and how to extract information at scale using advanced methodology.
🎩 H/T: Dmitry (Soxoj) Danilov
📺 Start Your Geolocation Search With Ease
Logan from Bellingcat demonstrates the usage of their latest OpenStreetMap search feature. This tool assists in pinpointing regions and specific areas for initiating geolocation searches, by identifying structures and characteristics present in images.
🎩 H/T: Logan Williams
🐦 Research Using Telegram’s Similar Channels
Gary shares his experience with the Similar Channels feature and toolset built by Soxoj while investigating terrorism networks. He shares a 23-page PDF demonstrating his findings.
🎩 H/T: Gary Warner
OSINT Tools
🌟 Sponsor: UserSearch.ai & Usersearch.org
Next-Level OSINT with UserSearch.ai
Unleash the full potential of your investigations with UserSearch 2.0, the all-in-one OSINT suite. Beyond usernames, our tool dives deep into emails, phones, images, and offers 24/7 target monitoring. Crafted over two years and trusted by experts, UserSearch.ai is your gateway to a world of comprehensive, detailed intelligence. Elevate your investigations, and experience the power of complete OSINT at your fingertips.
Sign in and browse our suite for free.
🔎 Porch Pirate
Porch Pirate evolved from a tool for uncovering Postman secrets to a comprehensive reconnaissance/OSINT framework. Use this tool to find new methods that are publicly exposed on Postman.
🎩 H/T: Mand Consulting Group
🖥️ Favicon Hash for Shodan
It’s possible to get the real IP address behind a Cloudflare-protected domain. You can do this by hashing the favicon for a website and searching that hash in IoT search engines like Censys or Shodan. This browser extension hashes the favicon for any website you’re on with the click of a button.
🎩 H/T: Michael Knap
🔎 Prying Deep
Several deep web crawlers on GitHub don't align with the required standards for deep web OSINT, prompting the creation of a specialized tool. This tool underscores the necessity of strict adherence to OPSEC best practices in the deep web environment. The developer of this tool disclaims any responsibility for potential harm resulting from its usage.
🎩 H/T: iudicium
✅ That’s it for the free version of The OSINT Newsletter. Consider upgrading to a paid subscription to support this publication and independent research.
By upgrading to paid, you’ll get access to the following:
⚡ Creating a facial recognition search engine from scratch
Collecting and verifying faces at scale from digital identifiers (email, username)
Encoding those faces without needing to store the images
Storing those encodings mapped to the digital identifiers
🚨 This is a 2 part series. In next week’s issue, I’ll finish this up by showing you how you can create a search engine to query this new dataset.
🗒️ This is for educational purposes only. You’ll read more about this later but my initial goal was to ‘reverse engineer’ how sites like PimEyes and Facecheck can produce this capability as a service.
👀 You get access to all paid posts in the archive. Go back and see what you’ve missed!
🚀 If you don’t have a paid subscription already, don’t worry there’s a 7-day free trial. If you like what you’re reading, upgrade your subscription. If you can’t, I totally understand. Stay tuned for the geolocation challenge in next week’s issue to get a shot at free access.
Keep reading with a 7-day free trial
Subscribe to The OSINT Newsletter to keep reading this post and get 7 days of free access to the full post archives.