A passive reconnaissance tool for known URLs discovery – it gathers a list of URLs passively using various online sources.
- Fetches known URLs from AlienVault’s OTX, Common Crawl, URLScan, Github and the Wayback Machine.
- Fetches disallowed paths from
robots.txtfound on your target domain and snapshotted by the Wayback Machine.
- Regex filter URLs.
- Save output to file.
You can download the pre-built binary for your platform from this repository’s releases page, extract, then move it to your
$PATHand you’re ready to go.
sigurlfind3r requires go1.14+ to install successfully. Run the following command to get the repo
GO111MODULE=on go get -u -v github.com/signedsecurity/sigurlfind3r/cmd/sigurlfind3r
git clone https://github.com/signedsecurity/sigurlfind3r.git && \ cd sigurlfind3r/cmd/sigurlfind3r/ && \ go build; mv sigurlfind3r /usr/local/bin/ && \ sigurlfind3r -h
sigurlfind3r will work after installation. However, to configure sigurlfind3r to work with certain services – currently github – you will need to have setup API keys. The API keys are stored in the
$HOME/.config/sigurlfind3r/conf.yaml file – created upon first run – and uses the YAML format. Multiple API keys can be specified for each of these services.
version: 1.4.0 sources: - commoncrawl - github - otx - urlscan - wayback - waybackrobots keys: github: - d23a554bbc1aabb208c9acfbd2dd41ce7fc9db39 - asdsd54bbc1aabb208c9acfbd2dd41ce7fc9db39
DiSCLAIMER: fetching urls from github is a bit slow.
This will display help for the tool.
_ _ __ _ _ _____ ___(_) __ _ _ _ _ __| |/ _(_)_ __ __| |___ / _ __ / __| |/ _` | | | | '__| | |_| | '_ \ / _` | |_ \| '__| \__ \ | (_| | |_| | | | | _| | | | | (_| |___) | | |___/_|\__, |\__,_|_| |_|_| |_|_| |_|\__,_|____/|_| 1.4.0 |___/ USAGE: sigurlfind3r [OPTIONS] OPTIONS: -d, --domain domain to fetch urls for -eS, --exclude-sources comma(,) separated list of sources to exclude -f, --filter URL filtering regex -iS, --include-subs include subdomains' urls -lS, --list-sources list all the available sources -nC, --no-color no color mode -s --silent silent mode: output urls only -uS, --use-sources comma(,) separated list of sources to use -o, --output output file
sigurlfind3r -d tesla.com
sigurlfind3r -d tesla.com -iS
sigurlfind3r -d tesla.com -f ".(jpg|jpeg|gif|png|ico|css|eot|tif|tiff|ttf|woff|woff2)"