find HIDDEN urls!! (subdomain enumeration hacking) // ft. HakLuke

About this video

An introduction to subdomain enumeration and hidden-URL discovery as recon, with a strong permission warning: only scan domains you own or that are in scope, for example through bug bounty programs on HackerOne for companies he names such as Snapchat, Tinder, Spotify and Reddit. A subdomain such as learn under his own domain can point to a completely separate server, and forgotten subdomains are a risk owners should find before attackers do. He installs and demos two Docker-based tools on Linux. HakCrawler, created by HakLuke, is the active option: it spiders a site and follows links page to page, also drawing on the Wayback Machine, robots.txt and sitemap files, and surfaces JavaScript files and subdomains with its subs option. Luke explains he wrote it in Go for native concurrency and compiled speed after knowing Python, mainly to automate a repetitive task for himself. The second tool, GAU, short for Get All URLs, is passive: it does not hit the target server but queries existing sources including AlienVault’s Open Threat Exchange, the Wayback Machine and Common Crawl. The broader message is to build your own tools even if similar ones exist, fork and share improvements, and not fear releasing imperfect code. A Private Internet Access VPN sponsorship is included mid-video.

Key takeaways

• Subdomain enumeration is recon: subdomains can point to entirely different servers, expanding the targets to audit or, in scope, to test
• Permission and scope come first; bug bounty programs define what may be scanned, and not every scope allows active enumeration
• HakCrawler does active crawling by spidering links, the Wayback Machine, robots.txt and sitemaps, and was written in Go chiefly for concurrency
• GAU (Get All URLs) is passive, pulling known URLs from sources such as AlienVault Open Threat Exchange, the Wayback Machine and Common Crawl instead of hitting the server
• Both tools were built by their authors to scratch their own itch, and he encourages viewers to build, fork and publicly share their own versions

Episode details: published July 25, 2022 · 17:30 long. Summary written by Self Growth Videos from the episode itself. Business results and any income figures mentioned are NetworkChuck’s own claims, shared for education — they are not typical or guaranteed.

← Previous: What is a Subnet Mask??? (you NEED to know it!!) · Next: Docker networking is CRAZY!! (you NEED to learn it) →

All NetworkChuck videos · NetworkChuck profile · NetworkChuck on YouTube