Accuracy is not the finish line

My dissertation started with a slightly uncomfortable observation. Almost every paper I read on financial forecasting ended the same way: a table of error scores, a bold number in the best row, and a conclusion that the new model was better. Very few asked the question a trader would ask first. Would this have made money?

So I built my research around that question. I trained GRUs, LSTMs, Transformers and a handful of hybrids on twenty-one years of hourly EUR/USD data, and then I made them trade.

The best scorer was not the best trader

A standalone Transformer posted one of the highest R² scores for close-price prediction. On paper it was a winner. But when I looked at consistency across folds and at how its errors behaved in volatile hours, the GRU-LSTM hybrid was the more trustworthy partner. It combined the GRU's quick reaction to short-term moves with the LSTM's longer memory.

A metric tells you how wrong a model is on average. A business needs to know when it will be wrong, and what that costs.

Split the job

Deep networks were surprisingly poor at calling market direction, barely better than a coin toss. XGBoost, a far less fashionable model, got it right almost every time, and stayed reliable even after I removed its strongest features to check it was not cheating. The final system gave each model the job it was best at. One decided direction, one estimated price, and the Kelly Criterion decided how much to stake.

Give good models room to be right

Short three- and four-hour trades lost money: there was simply too much noise. Held for twenty-four hours, the same signals delivered a 29% return on data the model had never seen. I think about that result constantly in client work. Often the model is fine and the decision window is wrong.

If you are evaluating an AI system, ask for the business metric next to the technical one. If nobody can produce it, that is your first project.