Saturday, April 20, 2013

More Thoughts On High Level System Requirements

I've started with three simultaneous projects:

  • Building a backtester that can use both end-of-day (EOD) data and 15 minute data.
    • Using java for the algo components
    • EOD data from
    • 15-minute data from
    • The data was loaded into a MySQL database with one table per symbol
      • The 15-minute data is requiring between 10GB and 25GB per table for five years of data.
      • The EOD data is requiring between 100MB and 700MB per table for two to five years of data
      • Also created tables:
        • containing the dates of market holidays -- will need to update this table annually -- possibly
        • containing options expiration dates for indexes for the last five years
      • Besides the MySQL Workbench, I am also using HeidiSQL and Navicat for MySQL.

  • Building a proof of concept (POC) of the end-to-end infrastructure required to run a live trading algorithm.  So far, I have built components to test
    • SSL from the browser to Node.js
    • Client certificates
    • Browser authentication with Node.js using the Passport.js module integrating with MySQL
    • Streaming data from a java process to RabbitMQ to Node.js to a Dojo table in a browser.
    • Java polling for data from Yahoo
    • Java streaming data via a DDE connection from ThinkOrSwim 

  • Building the trading system.
    • Just in the design phase at this point.  Will need at least the following components:
      • User Interface (Node.js)
      • Strategy Manager
      • Position Manager
      • Risk Manager
      • Broker Connector (streaming and FIX)
      • DB Connector (to MySQL)
      • Order Manager
      • Account Manager
      • Communications Manager
      • Message Bus - (RabbitMQ or similar)