Node, Rabbit, Ubuntu Tech POC

For the machine hosting Node.js and RabbitMQ, I am using Ubuntu Linux 12.04 LTS Server 32-bit.  Here are the high level configuration steps:

1. Download and install Ubuntu (http://www.ubuntu.com/download/server) on the target machine.
2. Install the Gnome Classic desktop.
3. Configure a static IP address for your server...I found this link helpful:
   https://help.ubuntu.com/12.04/serverguide/network-configuration.html
4. Configure VNC...this link was helpful:
   http://rbgeek.wordpress.com/2012/06/25/how-to-install-vnc-server-on-ubuntu-server-12-04/ 
5. Configure FTP...this link was helpful:
   http://en.kioskea.net/faq/7197-installing-an-ftp-server-under-ubuntu
6. Install Node.js...here's the link:
   https://github.com/joyent/node/wiki/Installing-Node.js-via-package-manager
7. Install RabbitMQ:
• Update Erlang...it should already be installed on Ubuntu:
  http://www.rabbitmq.com/which-erlang.html
  http://www.erlang.org/download.html
• Install RabbitMQ:
  http://www.rabbitmq.com/install-debian.html
8. Secure the server:
• http://www.thefanclub.co.za/how-to/how-secure-ubuntu-1204-lts-server-part-1-basics
• https://www.net24.co.nz/kb/article/AA-00227
• http://notepad2.blogspot.com/2012/03/binbash-flush-all-chainsiptables-f-set.html
• http://www.thegeekstuff.com/2011/06/iptables-rules-examples/
• http://wiki.xtronics.com/index.php/IP_Subnet_Masks
• I created an iptables bash script, using the third bullet above as a model.

That's about it ...

And at this point it cannot be accessed from the internet...only on the LAN.

More Thoughts On High Level System Requirements

I've started with three simultaneous projects:

• Building a backtester that can use both end-of-day (EOD) data and 15 minute data.

• Using java for the algo components
• EOD data from http://www.ivolatility.com/
• 15-minute data from http://www.livevol.com/
• The data was loaded into a MySQL database with one table per symbol
• The 15-minute data is requiring between 10GB and 25GB per table for five years of data.
• The EOD data is requiring between 100MB and 700MB per table for two to five years of data
• Also created tables:
• containing the dates of market holidays -- will need to update this table annually -- possibly
• containing options expiration dates for indexes for the last five years
• Besides the MySQL Workbench, I am also using HeidiSQL and Navicat for MySQL.

• Building a proof of concept (POC) of the end-to-end infrastructure required to run a live trading algorithm.  So far, I have built components to test

• SSL from the browser to Node.js
• Client certificates
• Browser authentication with Node.js using the Passport.js module integrating with MySQL
• Streaming data from a java process to RabbitMQ to Node.js to a Dojo table in a browser.
• Java polling for data from Yahoo
• Java streaming data via a DDE connection from ThinkOrSwim 

• Building the trading system.

• Just in the design phase at this point.  Will need at least the following components:
• User Interface (Node.js)
• Strategy Manager
• Position Manager
• Risk Manager
• Broker Connector (streaming and FIX)
• DB Connector (to MySQL)
• Order Manager
• Account Manager
• Communications Manager
• Message Bus - (RabbitMQ or similar)

Thoughts On High Level Trading System Requirements

Some thoughts on very high-level requirements...

• I'd like to be able to be able to monitor, modify and run this system from a browser...preferably a browser on my tablet.

• Will need to stream real-time algorithm status ...trades, current P&L, risk, etc...to a browser.

• Technology thoughts:
• Use SSL, client certificates, and userid/password to login to the system.
• Use the Dojo library for browser based, updating tables.
• Use Node.js to stream realtime updates to a browser...using websockets/socket.io.
• Use RabbitMQ to send data to Node.js...amqp.
• Use multiple java components (possibly running as individual processes) as the core of the system.  These components will send/receive updates to/from Node.js via RabbitMQ.
• The java components will receive option data - TBD
• The java components will send/receive order information via FIX/QuickFIX/J
• The java components will receive account updates via FIX/QuickFIX/J or a broker API.