2023-11-24 14:56:35 -05:00
2023-08-06 23:36:41 -04:00
2023-06-09 22:40:35 -04:00
2026-05-07 23:45:07 -04:00
2023-06-09 22:40:35 -04:00
2023-06-09 22:40:35 -04:00
2023-06-09 22:40:35 -04:00
2023-06-17 10:10:14 -04:00

Description

A Nestjs and Typescript Bitcoin stratum mining server.

Installation

$ npm install

create an new .env file in the root directory and configure it with the parameters in .env.example

Running the app

# development
$ npm run start

# watch mode
$ npm run start:dev

# production build
$ npm run build

Test

# unit tests
$ npm run test

# test coverage
$ npm run test:cov

Web interface

See public-pool-ui

Deployment

The production Docker image runs database migrations and then starts PM2 automatically. For a manual host deployment, install pm2 (https://pm2.keymetrics.io/), run migrations, then start the app.

$ npm run build
$ npm run migration:run:prod
$ pm2 start dist/main.js

When running the worker app in PM2 cluster mode, start the PM2 daemon with OS-level connection scheduling. The environment variable must be present when the PM2 daemon starts, not only in the worker configuration.

$ NODE_CLUSTER_SCHED_POLICY=none pm2 start ecosystem.config.js

Cluster-mode connection dropping requires Node.js 22.12.0 or newer.

STRATUM_WORKERS=auto is the default. It uses the container's available CPU count after reserving API_WORKERS CPUs plus one for the master. A numeric value still pins an exact topology. Size fixed deployments from the busiest worker, not just total connection capacity: keep roughly 10,000 or fewer miners per worker when low new-tip fanout latency matters. The stratum_job_fanout overTargetClients field uses STRATUM_FANOUT_TARGET_CLIENTS_PER_WORKER to make an undersized worker tier visible. Pools beyond one host's practical CPU or socket capacity should shard listeners across multiple worker hosts connected to the same urgent Redis channel.

STRATUM_MAX_CONNECTIONS_PER_LISTENER is enforced per worker and Stratum port. Size it using the busiest port: worker count * limit. For example, 28 workers with the default limit of 10000 allow up to 280000 connections on one port.

SV2 pool-assigned extranonces reserve the first of their four prefix bytes for a process namespace and share the remaining 24-bit allocation space across standard and extended channels. Under PM2, the namespace uses NODE_APP_INSTANCE (falling back to pm_id) plus restart_time parity, which keeps workers and overlapping zero-downtime reload generations disjoint without changing the advertised prefix or total extranonce sizes. The effective value is SV2_EXTRANONCE_NAMESPACE_BASE + 2 * worker + (restart_time % 2) and must be at most 255 (up to 128 worker lanes at base zero). Startup fails if a PM2 worker identity is missing or the namespace cannot fit. Independent containers or PM2 worker apps that share mining work must be assigned non-overlapping base ranges; reserve two namespace values per configured Stratum worker.

New-block notification path

The master keeps an authoritative Bitcoin Core getblocktemplate longpoll open; hashblock ZMQ remains a watchdog and duplicate results are discarded. Optional endpoints in BITCOIN_RPC_AUX_URLS keep independent longpolls open and race the primary source, reducing dependence on one node's block-relay peers. An auxiliary template is only eligible after the primary Core's getbestblockhash exactly matches its previous block hash; a mismatch or unavailable primary fails closed before any urgent or canonical miner notification. On a new tip, the master publishes a compact subsidy-only SV1 job over dedicated urgent Redis publisher/subscriber connections. It waits only SV1_BRIDGE_PUBLISH_BUDGET_MS for acknowledgement before proceeding. A timeout immediately starts the same bridge on the independent normal Redis command socket, so a stuck urgent socket cannot suppress delivery or block canonical full-template publication.

After canonical publication, the master publishes a durable placeholder-prevhash empty template for the following height. Stratum workers replay it after restart and prebuild every connected miner's height- and payout-bound coinbase in batches of SV1_PRESTAGE_BATCH_SIZE, yielding between batches. A newly authorized miner also stages against the latest template. The next authoritative bridge promotes the same cached job, patches the authoritative header fields into its already serialized notify buffer, and writes it without rebuilding the coinbase or JSON. Canonical full work then follows as a second clean switch. Payout snapshot creation and Postgres persistence remain outside the urgent path.

SV2 solo channels pre-stage a native subsidy-only future job for the next height. When the authoritative header arrives, the pool activates that job with only SetNewPrevHash; the same-tip full job follows without a second prevhash switch. Standard and extended candidates retain exact header/body reconstruction, and late network-target candidates remain recoverable without crediting stale shares. Pending SV2 canonical jobs are coalesced per client, while a new-tip activation is moved ahead of any not-yet-started canonical work for that tip. The finite defaults are 16 retained jobs per channel, four queued operations, 256 KiB of outstanding socket writes, and a two-second write-callback deadline. A client is disconnected if SV2_MAX_RETAINED_JOBS_PER_CHANNEL, SV2_MAX_QUEUED_JOB_OPERATIONS, SV2_MAX_SOCKET_BUFFER_BYTES, or SV2_SOCKET_WRITE_TIMEOUT_MS is exceeded; SV2_JOB_RETENTION_MS controls how long stale network candidates remain reconstructable.

SV1_SUBSIDY_BRIDGE_ENABLED=true enables the solo bridge (the default). PPLNS is never allowed to fall back to a miner-address coinbase; optional PPLNS bridge support requires a precomputed subsidy-valued payout snapshot. Retained jobs are kept for STRATUM_JOB_RETENTION_MS so a late network-target candidate can still be reconstructed and submitted, while ordinary old-tip shares are rejected. PPLNS seeds use a non-active snapshot status and are skipped if the next authoritative nBits differs from their preparation basis. Before any bridge is published, the master verifies that Core's coinbasevalue equals the locally calculated consensus subsidy plus every GBT transaction fee. Master startup also fails if NETWORK does not match Core's reported chain.

The Redis protocol remains rolling-deploy compatible: new workers retain the legacy mining-info reload path, while the master writes the historical latest key as JSON only after a PPLNS-safe compatibility template is ready. Deploying workers before the master is still the preferred rollout order. When any PPLNS listener is configured and snapshot preparation fails, legacy workers are held on their prior job instead of being woken with a miner-address fallback job.

Two structured log events expose the end-to-end timing:

  • block_notification_trace reports Core, bridge, Redis, PPLNS, and persistence stages, separated by payout mode and job type.
  • stratum_job_fanout reports true source-to-fanout-start time, master publish, worker receipt/handling, prestage hits, client count, bytes, backpressure, and p50/p95/p99/last enqueue time, correlated by eventId.
  • sv1_job_prestage reports the number of miners prepared for the next height and the background preparation duration.

For upstream latency, place the Core nodes in different well-connected networks, enable normal compact-block relay, and keep Redis and Stratum workers close together. Auxiliary nodes remain untrusted candidates for tip detection: the primary Core authorizes their exact tip before publication. Do not point BITCOIN_RPC_AUX_URLS at third-party RPC services.

Docker

The default compose stack includes Public Pool, TimescaleDB, and Redis. TimescaleDB stores normal Postgres tables plus immutable accepted-share rows in a hypertable. Chart and hashrate APIs read from Timescale continuous aggregates plus recent raw shares for realtime buckets. Redis is used only for process messaging and latest mining-template replay.

Start the stack:

$ docker compose up --build -d

Use an external TimescaleDB/Postgres server:

$ DB_HOST=postgres.example.com \
  DB_PORT=5432 \
  DB_USERNAME=public_pool \
  DB_PASSWORD='change-me' \
  DB_DATABASE=public_pool \
  DB_SSL=false \
  docker compose -f docker-compose.external-db.yml up --build -d

The external database must be TimescaleDB-compatible and reachable from the Public Pool container. The startup script waits for the remote DB_HOST:DB_PORT, runs migrations, then starts PM2. Redis still runs locally in this compose file unless REDIS_URL is pointed at an external Redis instance.

Set DB_SSL=true if the external database requires TLS. For private CA or self-signed test deployments, DB_SSL_REJECT_UNAUTHORIZED=false disables certificate verification; do not use that setting for normal internet-facing production databases.

For a fresh external database, create the role/database first:

CREATE USER public_pool WITH PASSWORD 'change-me';
CREATE DATABASE public_pool OWNER public_pool;
\c public_pool
CREATE EXTENSION IF NOT EXISTS timescaledb;
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";

If TimescaleDB is managed by another operator, make sure the public_pool role can create tables, indexes, continuous aggregates, and Timescale policies in the target database.

Run only the database migrations:

$ docker compose run --rm public-pool npm run migration:run:prod

Watch logs:

$ docker compose logs --tail 100 -f public-pool

Back up TimescaleDB:

$ docker compose exec timescaledb pg_dump -U public_pool public_pool > public-pool.sql

Redis does not hold durable accounting data. Losing Redis requires workers to replay the latest template from Redis after reconnect or fall back to the saved RPC block template table.

note: To successfully connect to the bitcoin RPC you will need to add

rpcallowip=172.16.0.0/12
zmqpubhashblock=tcp://0.0.0.0:3000

to your bitcoin.conf.

Testing

Baseline unit regression capture:

$ npm run test:baseline

Unit tests:

$ npm test

Integration tests against real TimescaleDB and Redis:

$ docker compose -f docker-compose.test.yml up --build --abort-on-container-exit

With the full-setup regtest Bitcoin Core running, validate reconstructed empty and full blocks through BIP23 proposal mode:

$ RUN_BITCOIN_REGTEST_INTEGRATION=true \
  npm run test:integration -- bitcoin-regtest-proposal
S
Description
No description provided
Readme
2.5 MiB
Languages
TypeScript 99%
Dockerfile 0.9%