A deep learning model combining Transformer and LSTM architectures to predict S&P 500 price movements using multiple economic indicators. The model leverages both global context understanding and sequential pattern recognition for predictions. Moveover, gradient is used to analysis the influence of different indicator. (For demonstration purposes, this framework is designed with a simplified example using monthly aggregated data)
main.py: Main script for training and evaluating the modelmodel.py: Contains the TimeTransformerLSTM model implementationexplanation.py: Functions for model interpretability and feature importance analysisutilities.py: Helper functions for data processing and visualization
data/: Contains the raw data and enhanced raw data (including time features and lag features)img/: Stores visualization outputs and model architecture diagramssaved_models/: Directory for storing trained model checkpoints
The hybrid architecture consists of:
- Input Processing: Handles multi-feature time series data with sliding windows
- Transformer Encoder: Captures global context and parallel processing within windows
- LSTM Layers: Processes enriched features while maintaining temporal order
- Output Layer: Generates predictions for specified time horizons
.png)
Key indicators include:
- Market Indicators: S&P 500 Price and Volume, BTC, Gold, Crude Oil
- Economic Indicators: (Source: FRED)
- EFFR (Effective Federal Funds Rate)
- CPI (Consumer Price Index)
- Treasury Yields (3m, 10y)
- GDP, Imports/Exports
- Risk Indicators: (Source: FRED)
- HY Index (U.S. High Yield Index) (market speculation)
- HPI (Home Price Index)
- DTI (Debt-to-Income Ratio)
- Contextual and Political Indicators::
- GEU (Global Economic Policy Uncertainty Index)
- Party (Ruling political party in the United States) (Custom Processed)
- Strong tracking of overall market trends
- Effective capture of long-term price movements
- Reasonable prediction accuracy in various market conditions
- Temporal importance varies across different features
- Some features show stronger predictive power at specific time steps (HY, PPI, UR, BTC, year_offset)
- Complex interactions between economic indicators revealed
- matplotlib==3.9.2
- numpy==2.0.2
- pandas==2.2.3
- scikit-learn==1.5.2
- torch==2.5.1+cu118