Trading tools
This study follows a series of personal tools written around MetaTrader 5, from a 124-line relay to a pipeline that collects the candles of a terminal, sends them to an API written in Go and keeps them in TimescaleDB.
- Role
- Sole author
- Period
- From March 2025 to August 2026
- Status
- In development. Personal tooling, with no third-party user
- Technologies
- Go, TimescaleDB, Python, MetaTrader 5, MQL5, Flask
- Repositories
- Three, plus a few standalone files
Context
This is tooling, not a method: no trading result is reported here. The series begins in April 2025 with a 124-line Flask relay, which keeps the last message received for each symbol and serves it back to a small Flutter application that polls it every fifteen seconds. That relay computes nothing and places no order, so the name “bot” carried by its repository suited it poorly.
A second repository follows in December 2025, which gathers a Python program driving the terminal and its equivalent written in MQL5, kept aligned by convention alone, together with a market scanning tool. The data pipeline dates from June 2026: it grew out of that scanning tool, which it serves from a database.
Problem
The candles sit in a terminal that runs at home, while the tools that read them are on a server. The candle in progress changes until it closes, and must therefore be resendable without creating a duplicate. The terminal also stamps its data in the time of the broker’s server, which is neither universal time nor local time and which shifts with daylight saving. Finally, commands had to reach a program located at home without opening a port on that network.
Answer
A natural key and an idempotent write
A candle is identified by its symbol, its timeframe and its opening time in universal time. The write updates the high, the low, the close and the volume of an existing row, never its open, so the collector resends the candle in progress on every cycle with no special care. Retention does not rest on a retention policy: the write and the prune run in the same transaction and keep a fixed number of rows per series.
const upsertSQL = `
INSERT INTO bars (symbol, timeframe, ts, open, high, low, close, volume)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8)
ON CONFLICT (symbol, timeframe, ts) DO UPDATE
SET high = EXCLUDED.high,
low = EXCLUDED.low,
close = EXCLUDED.close,
volume = EXCLUDED.volume;`
// pruneSQL deletes rows older than the window-th newest bar for a series, keeping
// exactly `window` rows. $3 is the zero-based offset of that row (window - 1).
// If fewer than `window` rows exist the subquery returns NULL and nothing is deleted.
const pruneSQL = `
DELETE FROM bars
WHERE symbol = $1 AND timeframe = $2
AND ts < (
SELECT ts FROM bars
WHERE symbol = $1 AND timeframe = $2
ORDER BY ts DESC
OFFSET $3 LIMIT 1
);`A buffer removed after 84 minutes
The first version of the collector kept in a SQLite database the pushes that had not gone through. It was removed 84 minutes after being written, since the terminal already keeps the history: the collector now asks the server what it holds and sends only what is missing, or the whole window on first start. A push that fails simply puts the series back into a checking state.
The broker’s clock treated as suspect
A fixed time offset, written in the configuration, stamped every candle one hour ahead for a while, after a switch to daylight saving time that went unnoticed. The offset has since been measured on every cycle, from a quote the collector checks to be recent, then rounded to the half hour. A candle dated in the future is refused by the collector, then again by the server, so that an offset error shows up as a refusal and not as wrong data. Storage stays in universal time and local time only appears at display.
At-most-once commands
The program that places orders and the page that drives it both connect, outbound, to a meeting point on the server, which removes the need to open a port at home. A command is a database row, which goes from pending to dispatched, then to success or failure. A command pending for more than five seconds is declared failed, and a command dispatched without acknowledgement is never replayed, the risk of an order placed twice outweighing that of an order lost.
Limits and outlook
The tests cover only three indicator functions and the display of the pages; nothing checks the write, the prune, the alerts or the command channel, and no pipeline runs these tests. The migrations are moreover replayed in full on every start, without a tracking table, which has already put the API in a restart loop. The database uses only the partitioned table of TimescaleDB, with no continuous aggregate and no compression, and gap detection in the series remains out of scope.
The documentation has fallen behind the code: it still describes the SQLite buffer and a time offset set by hand. In the previous repository, the two implementations kept aligned by convention ended up diverging on several rules. The series also has its dead ends, namely the relay and its application, which were never reused, and a panel mock-up that was never wired to the terminal.