chore(receiver): packaging, k8s manifests, CI/release, docs
Dockerfile.receiver (CGO-free, /data volume), receiver image in CI and tag releases, receiver binary in release archives, make build-receiver, example k8s manifests (deployment/service/ingress/secret/pvc) under deploy/k8s/receiver/, and README coverage including sender/receiver token pairing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Gary Hansen
co-authored by
Claude Fable 5
Claude Fable 5
parent
beb595442f
commit
c9a963ecdd
@@ -297,6 +297,7 @@ variables at startup:
|
||||
| `EXPLOREDNS_CORS_ORIGIN` | *(unset)* | Off by default (the SPA is same-origin). Set an origin — or `*` for development — to enable cross-origin API access. |
|
||||
| `EXPLOREDNS_RATE_LIMIT` | `30/1h` | Per-client-IP token-bucket limit on `POST /api/traverse` in `N/duration` form (e.g. `10/10m`); invalid values fall back to the default. Over-limit requests get `429`. Buckets refill continuously. Direct localhost connections are exempt (dev loop, tests), but proxied requests are always limited by the real client IP from `Fly-Client-IP` / `X-Forwarded-For`. |
|
||||
| `EXPLOREDNS_WEBHOOK_URL` | *(unset)* | Off by default. When set, the server POSTs a usage-reporting JSON event to this URL on every traversal start and completion (see below). |
|
||||
| `EXPLOREDNS_WEBHOOK_TOKEN` | *(unset)* | Optional bearer token for webhook deliveries. When set, every webhook POST carries `Authorization: Bearer <token>`; pair it with the receiver's `RECEIVER_INGEST_TOKEN`. |
|
||||
|
||||
### Usage reporting
|
||||
|
||||
@@ -312,12 +313,17 @@ traversal, each with header `X-ExploreDNS-Event` naming the event:
|
||||
`X-Forwarded-For` entry, else the connection address). Delivery is
|
||||
fire-and-forget: a 5-second timeout, one retry after 2 seconds, and failures
|
||||
are logged without ever affecting the traversal or the API response. On
|
||||
Fly.io, configure it as a secret rather than in `fly.toml`:
|
||||
Fly.io, configure it (and the optional bearer token) as secrets rather than
|
||||
in `fly.toml`:
|
||||
|
||||
```sh
|
||||
fly secrets set EXPLOREDNS_WEBHOOK_URL=https://example.com/hook
|
||||
fly secrets set EXPLOREDNS_WEBHOOK_URL=https://example.com/hook \
|
||||
EXPLOREDNS_WEBHOOK_TOKEN=some-long-random-string
|
||||
```
|
||||
|
||||
This repo ships a matching receiver for these events — see
|
||||
[Usage telemetry receiver](#usage-telemetry-receiver).
|
||||
|
||||
---
|
||||
|
||||
## Deploying to Fly.io
|
||||
@@ -366,9 +372,9 @@ flyctl tokens create deploy -x 999999h
|
||||
|
||||
Pushing a `v*` tag triggers the full release pipeline:
|
||||
|
||||
1. `.gitea/workflows/release.yml` (`binaries` job) cross-compiles the CLI
|
||||
and server for linux/amd64, linux/arm64, darwin/amd64, darwin/arm64 and
|
||||
windows/amd64, packages them as
|
||||
1. `.gitea/workflows/release.yml` (`binaries` job) cross-compiles the CLI,
|
||||
server, and receiver for linux/amd64, linux/arm64, darwin/amd64,
|
||||
darwin/arm64 and windows/amd64, packages them as
|
||||
`exploredns_<tag>_<os>_<arch>.tar.gz` (`.zip` on Windows) plus a
|
||||
`SHA256SUMS` file, and attaches everything to the Gitea release for the
|
||||
tag. Create the release with notes by hand before (or after) pushing
|
||||
@@ -376,8 +382,8 @@ Pushing a `v*` tag triggers the full release pipeline:
|
||||
a bare one only when none exists, and skips already-attached assets so
|
||||
re-runs are safe.
|
||||
2. `.gitea/workflows/release.yml` (`docker` job) pushes
|
||||
`gitea.hansenits.com.au/hits/exploredns-cli` and `…/exploredns-web`
|
||||
images tagged `<tag>` and `latest`.
|
||||
`gitea.hansenits.com.au/hits/exploredns-cli`, `…/exploredns-web`, and
|
||||
`…/exploredns-receiver` images tagged `<tag>` and `latest`.
|
||||
3. `.gitea/workflows/deploy.yml` deploys the web server to Fly.io.
|
||||
|
||||
All binaries are stamped with the tag via
|
||||
@@ -420,6 +426,78 @@ with `--show-servers`. Suitable for piping into `jq`.
|
||||
|
||||
---
|
||||
|
||||
## Usage telemetry receiver
|
||||
|
||||
`cmd/exploredns-receiver` is a small companion service that receives the
|
||||
usage webhooks described above (`start`/`complete` events from
|
||||
`EXPLOREDNS_WEBHOOK_URL`), stores them in MySQL or SQLite, and serves a
|
||||
basic-auth-protected admin dashboard (`/admin`) plus JSON API
|
||||
(`/admin/api/traversals`, `/admin/api/stats`) over the collected data. It
|
||||
is a separate binary intended to run wherever you keep long-lived storage
|
||||
(e.g. a home Kubernetes cluster) while the public web server stays
|
||||
stateless.
|
||||
|
||||
Endpoints: `POST /webhook` (ingest, bearer-token protected when configured),
|
||||
`GET /healthz` (liveness/readiness), `GET /admin` and `GET /admin/api/*`
|
||||
(basic auth, always required).
|
||||
|
||||
### Configuration
|
||||
|
||||
| Variable | Default | Meaning |
|
||||
|---|---|---|
|
||||
| `RECEIVER_ADDR` | `:8080` | Listen address. |
|
||||
| `RECEIVER_MYSQL_DSN` | *(unset)* | [go-sql-driver DSN](https://github.com/go-sql-driver/mysql#dsn-data-source-name) (`user:pass@tcp(host:3306)/dbname`). When set, events are stored in MySQL and the SQLite settings are ignored. |
|
||||
| `RECEIVER_SQLITE_PATH` | `data/exploredns-receiver.db` | SQLite database path, used when no MySQL DSN is set (the container image defaults it to `/data/exploredns-receiver.db`). Parent directories are created automatically. |
|
||||
| `RECEIVER_INGEST_TOKEN` | *(unset)* | When set, `POST /webhook` requires `Authorization: Bearer <token>`. Leave unset only on trusted networks. |
|
||||
| `RECEIVER_ADMIN_USER` | `admin` | Basic-auth username for `/admin`. |
|
||||
| `RECEIVER_ADMIN_PASSWORD` | *(required)* | Basic-auth password for `/admin`; the receiver refuses to start without it. |
|
||||
|
||||
Both storage backends share one portable schema; pure-Go drivers
|
||||
(`modernc.org/sqlite`, `github.com/go-sql-driver/mysql`) keep the binary
|
||||
CGO-free. SQLite is the zero-setup default; point `RECEIVER_MYSQL_DSN` at
|
||||
an external MySQL when you want the data outside the pod/VM.
|
||||
|
||||
### Running with Docker
|
||||
|
||||
```sh
|
||||
docker run -d --name exploredns-receiver \
|
||||
-p 8080:8080 \
|
||||
-v exploredns-receiver-data:/data \
|
||||
-e RECEIVER_ADMIN_PASSWORD=change-me \
|
||||
-e RECEIVER_INGEST_TOKEN=some-long-random-string \
|
||||
gitea.hansenits.com.au/hits/exploredns-receiver:latest
|
||||
```
|
||||
|
||||
The image stores SQLite data under the `/data` volume; add
|
||||
`-e RECEIVER_MYSQL_DSN=...` to use MySQL instead.
|
||||
|
||||
### Running on Kubernetes
|
||||
|
||||
[deploy/k8s/receiver/](deploy/k8s/receiver/) contains commented template
|
||||
manifests: a single-replica deployment (SQLite on a 1Gi PVC mounted at
|
||||
`/data`, probes on `/healthz`), ClusterIP service, ingress with TLS
|
||||
placeholders, and a secret template for the `RECEIVER_*` variables. Edit
|
||||
the placeholder host/credentials, then:
|
||||
|
||||
```sh
|
||||
kubectl apply -f deploy/k8s/receiver/
|
||||
```
|
||||
|
||||
Keep one replica while on SQLite; MySQL removes that constraint.
|
||||
|
||||
### Pairing with the web server
|
||||
|
||||
Set the same token on both ends so the receiver only accepts events from
|
||||
your server — e.g. on Fly.io:
|
||||
|
||||
```sh
|
||||
fly secrets set EXPLOREDNS_WEBHOOK_URL=https://receiver.example.com/webhook \
|
||||
EXPLOREDNS_WEBHOOK_TOKEN=some-long-random-string
|
||||
# receiver side: RECEIVER_INGEST_TOKEN=some-long-random-string
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Comparison with dnstraverse
|
||||
|
||||
| Feature | dnstraverse (Ruby) | ExploreDNS (Go) |
|
||||
@@ -440,13 +518,16 @@ with `--show-servers`. Suitable for piping into `jq`.
|
||||
```
|
||||
cmd/exploredns/ CLI entry point and flag parsing
|
||||
cmd/server/ HTTP API server entry point
|
||||
cmd/exploredns-receiver/ Usage telemetry receiver entry point
|
||||
internal/config/ Configuration types, validation, and usage text
|
||||
internal/dns/ DNS query layer, root discovery, transport
|
||||
internal/traverse/ Core traversal engine, referral resolution, caching
|
||||
internal/fingerprint/ DNS server version fingerprinting (version.bind CHAOS)
|
||||
internal/output/ Result formatting — text tree and JSON renderers
|
||||
internal/integration/ End-to-end integration tests
|
||||
internal/receiver/ Telemetry receiver: HTTP server, admin UI, event store
|
||||
web/api/ HTTP handler, job store, SSE streaming, static assets
|
||||
deploy/k8s/receiver/ Kubernetes manifest templates for the receiver
|
||||
```
|
||||
|
||||
---
|
||||
@@ -456,7 +537,8 @@ web/api/ HTTP handler, job store, SSE streaming, static assets
|
||||
```sh
|
||||
make build # compile CLI binary to bin/exploredns
|
||||
make build-server # compile server binary to bin/exploredns-server
|
||||
make build-all # compile both binaries
|
||||
make build-receiver # compile telemetry receiver to bin/exploredns-receiver
|
||||
make build-all # compile all three binaries
|
||||
make test # run all unit and integration tests
|
||||
make lint # run go vet
|
||||
make clean # remove build artefacts
|
||||
|
||||
Reference in New Issue
Block a user