Gary HansenandClaude Fable 5
ccfd3fd156
fly.toml (Dockerfile.web, scale-to-zero in syd, /api/health checks), make deploy / deploy-status targets, a Gitea workflow deploying on v* tags or manual dispatch via FLY_API_TOKEN, and README instructions covering first-time setup and the server's runtime env knobs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
416 lines
14 KiB
Markdown
416 lines
14 KiB
Markdown
# ExploreDNS
|
||
|
||
ExploreDNS is a comprehensive DNS traversal tool that explores every possible
|
||
resolution path for a domain — from the root servers all the way down — just
|
||
like a real iterative resolver, but without stopping at the first answer. It
|
||
follows every referral exhaustively, collates all results, and presents them in
|
||
a structured, human-readable (or JSON) report.
|
||
|
||
Inspired by the classic [dnstraverse](https://github.com/squish/dnstraverse)
|
||
Ruby tool, ExploreDNS is a modern Go rewrite that produces a self-contained
|
||
binary with no runtime dependencies.
|
||
|
||
---
|
||
|
||
## Features
|
||
|
||
- **Full iterative traversal** — queries root servers and follows every referral
|
||
branch, mirroring real resolver behaviour
|
||
- **All root servers** — optionally query all 13 root server sets in parallel
|
||
- **No-glue resolution** — automatically resolves nameserver addresses when
|
||
referrals lack glue records
|
||
- **DNS server fingerprinting** — identifies server software via `version.bind`
|
||
CHAOS queries
|
||
- **Multiple output formats** — coloured text tree and machine-readable JSON
|
||
- **Configurable transport** — UDP/TCP, EDNS0 buffer size, retries, timeouts
|
||
- **Fast mode** — shares glue across branches for speed; disable for independent
|
||
paths
|
||
- **CNAME tracking** — follows CNAME chains and detects loops
|
||
- **Web interface** — browser-based UI backed by an HTTP API server with
|
||
real-time Server-Sent Events progress streaming
|
||
|
||
---
|
||
|
||
## Installation
|
||
|
||
### Pre-built binary
|
||
|
||
Download the latest release binary for your platform from the
|
||
[Releases page](https://gitea.hansenits.com.au/hits/ExploreDNS/releases).
|
||
|
||
### Build from source
|
||
|
||
Requires Go 1.21 or later.
|
||
|
||
```sh
|
||
git clone https://gitea.hansenits.com.au/hits/ExploreDNS.git
|
||
cd ExploreDNS
|
||
make build # produces bin/exploredns
|
||
make build-server # produces bin/exploredns-server
|
||
make build-all # produces both binaries
|
||
```
|
||
|
||
### go install
|
||
|
||
```sh
|
||
go install gitea.hansenits.com.au/hits/ExploreDNS/cmd/exploredns@latest
|
||
go install gitea.hansenits.com.au/hits/ExploreDNS/cmd/server@latest
|
||
```
|
||
|
||
---
|
||
|
||
## Quick Start
|
||
|
||
```sh
|
||
# Basic A record traversal
|
||
exploredns www.example.com
|
||
|
||
# MX records for a domain
|
||
exploredns --type MX example.com
|
||
|
||
# Use all 13 root server sets
|
||
exploredns --all-root-servers www.example.com
|
||
|
||
# JSON output
|
||
exploredns --json www.example.com
|
||
|
||
# Quiet (results only)
|
||
exploredns --quiet www.example.com
|
||
|
||
# Debug mode
|
||
exploredns --debug www.example.com
|
||
|
||
# Debug plus library-level diagnostics
|
||
exploredns --dd www.example.com
|
||
|
||
# Force TCP
|
||
exploredns --allow-tcp --always-tcp www.example.com
|
||
|
||
# Disable fast mode (independent paths per branch)
|
||
exploredns --fast=false www.example.com
|
||
|
||
# Increase traversal depth limit
|
||
exploredns --max-depth 30 www.example.com
|
||
```
|
||
|
||
---
|
||
|
||
## CLI Reference
|
||
|
||
```
|
||
Usage:
|
||
exploredns [flags] <domain>
|
||
|
||
Query Options:
|
||
--type <TYPE> Record type to query (default: a)
|
||
Supported: A, AAAA, NS, CNAME, MX, TXT, SOA, PTR, ANY
|
||
--root-server <HOST> Initial root server, hostname or IP literal
|
||
(default: ask the upstream resolver for one root)
|
||
--all-root-servers Traverse from all root servers (default: false)
|
||
--root-aaaa Include IPv6 root addresses (not implemented yet)
|
||
--follow-aaaa Only follow AAAA for referrals (not implemented yet)
|
||
--dns-upstream <ADDR> Upstream resolver (host:port) for root discovery
|
||
(default: system resolver)
|
||
|
||
Transport Options:
|
||
--udp-size <N> EDNS0 UDP buffer size, 512–4096; 512 turns EDNS0 off
|
||
(default: 2048)
|
||
--allow-tcp Fall back to TCP on truncation (default: true)
|
||
--always-tcp Always use TCP (requires --allow-tcp)
|
||
--retries <N> Number of 2s retries before timing out, 0–10 (default: 2)
|
||
|
||
Traversal Options:
|
||
--max-depth <N> Maximum referral depth, 1–100 (default: 20)
|
||
--fast / --fast=false Fast mode; turn off to be more accurate (default: true)
|
||
|
||
Output Options:
|
||
--json Emit a single JSON document instead of text
|
||
--verbose, -v Verbose progress ([qname] and <bailiwick> shown)
|
||
-d, --debug Print debug diagnostics to stderr
|
||
-dd Like -d plus library-level debug
|
||
--quiet, -q Suppress the header block
|
||
--show-progress Show traversal progress (default: true)
|
||
--show-resolves Show glue-resolution progress (default: false)
|
||
--show-servers Show servers encountered (default: false)
|
||
--show-versions Show server version fingerprints (default: true)
|
||
--show-all-stats Show statistics after every node (default: false)
|
||
--show-results Show the results (default: true)
|
||
--show-summary-results Show the summary results (default: true)
|
||
```
|
||
|
||
Every `--show-X` flag can be negated with `--show-X=false` or `--no-show-X`.
|
||
|
||
---
|
||
|
||
## Web Interface
|
||
|
||
ExploreDNS ships a second binary — `exploredns-server` — that exposes a
|
||
browser-based UI and a JSON REST API backed by the same traversal engine as
|
||
the CLI.
|
||
|
||
### Starting the server
|
||
|
||
```sh
|
||
# Default: listen on :8080
|
||
./bin/exploredns-server
|
||
|
||
# Custom address
|
||
./bin/exploredns-server --addr :9090
|
||
./bin/exploredns-server --addr 127.0.0.1:8080
|
||
```
|
||
|
||
Or via Make:
|
||
|
||
```sh
|
||
make build-server
|
||
./bin/exploredns-server
|
||
```
|
||
|
||
Open `http://localhost:8080` in your browser. The SPA lets you enter a domain,
|
||
choose a record type, and watch the traversal progress in real time. When the
|
||
traversal completes the full result tree is displayed in the browser.
|
||
|
||
### API endpoints
|
||
|
||
| Method | Path | Description |
|
||
|--------|------|-------------|
|
||
| `POST` | `/api/traverse` | Start an asynchronous traversal |
|
||
| `GET` | `/api/traverse/{id}` | Poll traversal status and results |
|
||
| `GET` | `/api/traverse/{id}/stream` | Server-Sent Events live progress stream |
|
||
| `GET` | `/api/health` | Health check — returns `{"status":"ok"}` |
|
||
|
||
#### POST /api/traverse
|
||
|
||
Request body (JSON):
|
||
|
||
```json
|
||
{
|
||
"domain": "www.example.com",
|
||
"type": "A",
|
||
"all_roots": false
|
||
}
|
||
```
|
||
|
||
`type` defaults to `"A"` if omitted. `all_roots` queries all 13 root server
|
||
sets in parallel (equivalent to `--all-root-servers` in the CLI).
|
||
|
||
Response (`202 Accepted`):
|
||
|
||
```json
|
||
{
|
||
"id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
|
||
"status": "running"
|
||
}
|
||
```
|
||
|
||
#### GET /api/traverse/{id}
|
||
|
||
Returns a snapshot of the job including the full result list once complete.
|
||
`status` is one of `running`, `complete`, or `error`.
|
||
|
||
```json
|
||
{
|
||
"id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
|
||
"status": "complete",
|
||
"domain": "www.example.com",
|
||
"query_type": "A",
|
||
"started_at": "2024-01-01T12:00:00Z",
|
||
"done_at": "2024-01-01T12:00:02Z",
|
||
"results": [
|
||
{
|
||
"depth": 2,
|
||
"probability": 1.0,
|
||
"response_type": "Answer",
|
||
"server": "192.0.2.53:53",
|
||
"answers": ["www.example.com. 3600 IN A 93.184.216.34"]
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
#### GET /api/traverse/{id}/stream
|
||
|
||
An [SSE](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events)
|
||
stream of `ProgressEvent` objects, one per `data:` message. Past events
|
||
recorded before the client connected are replayed immediately, then live events
|
||
follow. The stream ends with `event: done`.
|
||
|
||
```
|
||
data: {"stage":"start","depth":1,"name":"www.example.com","qtype":"A","bailiwick":"com"}
|
||
|
||
data: {"stage":"complete","depth":1,"name":"www.example.com","qtype":"A","server":"192.0.2.53:53","bailiwick":"com"}
|
||
|
||
event: done
|
||
data: {}
|
||
```
|
||
|
||
Completed jobs are kept in memory for one hour before being purged.
|
||
|
||
### Server configuration
|
||
|
||
The server is safe to expose publicly by default and reads these environment
|
||
variables at startup:
|
||
|
||
| Variable | Default | Meaning |
|
||
|---|---|---|
|
||
| `EXPLOREDNS_JOB_TIMEOUT` | `5m` | Hard deadline per traversal (Go duration). Timed-out jobs report `error` with any partial results. |
|
||
| `EXPLOREDNS_MAX_JOBS` | `8` | Maximum concurrent traversals; further `POST /api/traverse` requests get `429`. |
|
||
| `EXPLOREDNS_CORS_ORIGIN` | *(unset)* | Off by default (the SPA is same-origin). Set an origin — or `*` for development — to enable cross-origin API access. |
|
||
|
||
---
|
||
|
||
## Deploying to Fly.io
|
||
|
||
The repo ships a [fly.toml](fly.toml) that builds `Dockerfile.web` and runs
|
||
the web server with scale-to-zero machines in `syd` (edit `app` /
|
||
`primary_region` to taste).
|
||
|
||
### First-time setup
|
||
|
||
```sh
|
||
flyctl auth login
|
||
flyctl apps create exploredns # match the app name in fly.toml
|
||
make deploy # flyctl deploy --remote-only
|
||
```
|
||
|
||
`make deploy-status` shows machine and health-check state. The app serves
|
||
the SPA at `https://<app>.fly.dev/` with `/api/health` as the health check.
|
||
|
||
### Continuous deployment
|
||
|
||
`.gitea/workflows/deploy.yml` deploys on any `v*` tag push (or manual
|
||
dispatch). It needs a `FLY_API_TOKEN` repository secret:
|
||
|
||
```sh
|
||
flyctl tokens create deploy -x 999999h
|
||
```
|
||
|
||
### Notes
|
||
|
||
- Traversal traffic is outbound UDP/TCP port 53, which Fly machines allow;
|
||
upstream root discovery uses Fly's internal resolver via `/etc/resolv.conf`
|
||
and falls back to the built-in IANA root hints.
|
||
- The job timeout, job cap, and same-origin CORS defaults above are what make
|
||
unauthenticated public exposure reasonable; tighten `EXPLOREDNS_MAX_JOBS`
|
||
if the app attracts traffic.
|
||
|
||
---
|
||
|
||
|
||
|
||
### Text (default)
|
||
|
||
dnstraverse-style output: a header block (settings, initial root, query;
|
||
suppressed by `--quiet`), progress lines (`<refid> <server> (<ips>)` with
|
||
` -- resolving` and ` -- completed earlier (<refid>)` markers), a `Results:`
|
||
section of aggregated outcomes with probabilities (`Answer from`, `No glue
|
||
at`, `Lame referral from`, error/exception wording), and a `Summary Results:`
|
||
section grouping outcomes by status and answer content. `--show-servers`
|
||
adds the sorted list of servers encountered. Colour is used only when stdout
|
||
is a terminal and the `NO_COLOR` environment variable is unset.
|
||
|
||
### JSON (`--json`)
|
||
|
||
A single JSON document — `{domain, qtype, root, results, summary, servers}` —
|
||
emitted once at the end of the run with deterministic ordering. Each
|
||
aggregated outcome appears exactly once in `results`; `summary` groups
|
||
probabilities by status and by distinct answer RRset; `servers` is present
|
||
with `--show-servers`. Suitable for piping into `jq`.
|
||
|
||
---
|
||
|
||
## Comparison with dnstraverse
|
||
|
||
| Feature | dnstraverse (Ruby) | ExploreDNS (Go) |
|
||
|---|---|---|
|
||
| Language | Ruby | Go |
|
||
| Self-contained binary | No | Yes |
|
||
| All root servers | Yes | Yes |
|
||
| No-glue resolution | Yes | Yes |
|
||
| JSON output | No | Yes |
|
||
| Server fingerprinting | Yes | Yes |
|
||
| CNAME loop detection | Partial | Yes |
|
||
| Active development | Dormant | Active |
|
||
|
||
---
|
||
|
||
## Project Structure
|
||
|
||
```
|
||
cmd/exploredns/ CLI entry point and flag parsing
|
||
cmd/server/ HTTP API server entry point
|
||
internal/config/ Configuration types, validation, and usage text
|
||
internal/dns/ DNS query layer, root discovery, transport
|
||
internal/traverse/ Core traversal engine, referral resolution, caching
|
||
internal/fingerprint/ DNS server version fingerprinting (version.bind CHAOS)
|
||
internal/output/ Result formatting — text tree and JSON renderers
|
||
internal/integration/ End-to-end integration tests
|
||
web/api/ HTTP handler, job store, SSE streaming, static assets
|
||
```
|
||
|
||
---
|
||
|
||
## Development
|
||
|
||
```sh
|
||
make build # compile CLI binary to bin/exploredns
|
||
make build-server # compile server binary to bin/exploredns-server
|
||
make build-all # compile both binaries
|
||
make test # run all unit and integration tests
|
||
make lint # run go vet
|
||
make clean # remove build artefacts
|
||
```
|
||
|
||
Run a single package's tests:
|
||
|
||
```sh
|
||
go test ./internal/traverse/...
|
||
```
|
||
|
||
---
|
||
|
||
## Architecture Overview
|
||
|
||
```
|
||
main → config.Parse → traverse.NewTraverser → traverse.Traverse
|
||
│
|
||
┌─────────▼──────────┐
|
||
│ Stack (BFS/DFS) │
|
||
│ Referral queue │
|
||
└─────────┬──────────┘
|
||
│ per referral
|
||
┌─────────▼──────────┐
|
||
│ processReferral │
|
||
│ ├─ resolveGlue │ (no-glue NS resolution)
|
||
│ └─ queryServer │ (dns.Query)
|
||
└─────────┬──────────┘
|
||
│
|
||
┌─────────────▼──────────────┐
|
||
│ Response classifier │
|
||
│ Answer / Referral / │
|
||
│ CNAME / NXDOMAIN / │
|
||
│ SERVFAIL / Error │
|
||
└─────────────┬──────────────┘
|
||
│
|
||
┌─────────▼──────────┐
|
||
│ output.Formatter │
|
||
│ (text | JSON) │
|
||
└────────────────────┘
|
||
```
|
||
|
||
The traversal engine (`internal/traverse`) maintains a work stack of
|
||
`Referral` objects. Each referral represents a single query to a single set
|
||
of nameservers. When a response contains further referrals, child `Referral`
|
||
objects are pushed onto the stack and processed in turn.
|
||
|
||
The `InfoCache` is used to store glue records discovered during traversal. In
|
||
fast mode (default), a single root cache is shared across all branches so that
|
||
glue discovered early is reused. In non-fast mode, each branch gets its own
|
||
independent cache.
|
||
|
||
---
|
||
|
||
## License
|
||
|
||
MIT — see [LICENSE](LICENSE).
|