Published On
Wed Oct 16th, 2013
 

A lot of tick data sources are available online and this is an attempt to cover some of the most well and popular providers where tick data are free to download. Dukascopy is considered the best of all due to the data quality and accessibility.

Dukascopy

Introduction

Dukascopy is known internationally as one of the best brokers nowadays. Many persons identify it as "The best ECN broker", and no bad behavior or attitude was noted from them until now. However their great obstacle is that they don't support an MT4 client, it would be very complicated to run MT4 EAs, but this is possible. So, if you need really solid ECN broker and not interested in Metatrader expert advisors, no need to search any more. Their JForex client supports extensive automation, but of course there wouldn't be as many commercial automated trading system for it as those available for Metatrader 4.

Dukascopy is considered the most popular tick data source as it offers free data spanning from April 2007 - for most currency pairs - until now. In spite of being of no cost, it's high quality and actually approaching the 99% quality present in the strategy tests - just a number in the FXT header - more over, it's hourly updated with no source except for the Dukascopy data feed server. In conclusion, you wouldn't need any better data quality.

Available currency pairs:

AUDCAD AUDCHF AUDJPY AUDNZD AUDUSD
CADCHF CADJPY CHFJPY EURAUD EURCAD
EURCHF EURGBP EURJPY EURNOK EURSEK
EURUSD GBPAUD GBPCAD GBPCHF GBPJPY
GBPNZD GBPUSD NZDCAD NZDCHF NZDJPY
NZDUSD USDCAD USDCHF USDDKK USDJPY
USDNOK USDSEK USDSGD XAGUSD XAUUSD

Downloading Tick Data

In the past downloading via Dukascopy historical data page was the most appropriate method where a web application could let you get the tick data for a certain time range is available, but nowadays it only allows downloading one day at a time, so it is no more a very convenient method.

  • Other options for downloading are:
  • The Tickstory is a free all-in-one tool that downloads the Dukascopy tick data and processes it into an FXT. If you select to use it, there would be no need for the CSV2FXT conversion step anymore besides that you are still able to export a CSV and do it, but unfortunately neither Tickstory nor Tickstory created FXT files are not supported in Tick Data Suite technical support.
  • The StrategyQuant Tick Data Downloader is a free easy to use application that automates tick data downloading and CSV creation
  • By means of JForex, the Dukascopy client. Detailed step are at Downloading Dukascopy tick data with the JForex client guide.
  • Using my PHP script collection, details are at Downloading and parsing the Dukascopy tick data with Birt's PHP scripts guide, this is only for tech-savvy persons that want to understand every single minute step in the tick data process.

It's of no importance which method you use, please take the time to read the "Purpose" and "No warranty by Dukascopy" sections on the Dukascopy tick data page.

Pros and cons

  • Pros:
  • Multiple download methods.
  • Extensive time interval coverage.
  • Wide array of available pairs.
  • Very good quality.
  • Cons:
  • Until November 2010, the data was not top-of-the-order-book, but rather aggregated in 0.5 pip increments.
  • The data for some pairs (minors and metals) doesn't go back near 2007.

Pepperstone and Integral

Introduction

Pepperstone is an ECN broker started in 2010, with a solid reputation and a growing client base. Many good things are traded on the tongues of many customers that have accounts there, being apparently there are no unsatisfied client. The tick data they offered by them is collected by their partner Integral, one of the leading liquidity providers. Their data is of high quality too, with generally lower spreads than that of Dukascopy data.

Their oldest data starts in 2009 and the available range of currency pairs is mostly limited to the majors as many minor currency pairs aren't there and no metals are present at all. The volume isn't included in the data.

Available currency pairs:

AUDJPY AUDNZD AUDUSD CADJPY CHFJPY
EURCHF EURGBP EURJPY EURUSD GBPJPY
GBPUSD NZDUSD USDCAD USDCHF USDJPY

Downloading Tick Data

The data are available for downloaded at the Pepperstone tick data page or at the TrueFX downloads page, the latter requires free registration.

  • The data is split in files, each of which is spanning for one month. To use it, you need to apply these steps:
  • Download the months of interest (should be consecutive).
  • Unpack all the zip files.
  • Concatenate the files in order. The best way to do that is:
    • Start the command prompt (Start->Run->cmd.exe)
    • Change to the directory where you unpacked the files - e.g. cd "\Documents and Settings\Administrator\My Documents\Downloads\"
    • Use the type command - e.g. type EURUSD-2011-01.csv EURUSD-2011-02.csv EURUSD-2011-03.csv EURUSD-2011-04.csv > EURUSD.csv

If you need more details about how to concatenate files, please Google it. As a side note, DownThemAll addon for Firefox browser can download all the data for the pair(s) you want without minimal clicking.

Pros and cons

  • Pros:
  • Very good quality.
  • Cons:
  • The first available date is 2009.
  • The data is split into months and the files have to be downloaded individually and manually concatenated (merged).
  • The pair name is included for every tick which is a complete waste of space and bandwidth

Onda

Introduction

Oanda is one of the oldest present brokers, established in 1996. Their Forex trading service with their proprietary trading platform started in 2001, their demo and watching the news on Nov 9th. Now, Oanda has an excellent reputation as a US-based market maker broker. Their platform greatest advantage was the ability to start trading with any amount you want as low as only $1, but after a MT4 platform was added there recently withe lot size was 100k they effectively lost that advantage. They're also quite famous for their very competitive interest rates which are calculated per second; no swap can be seen in the fxTrade platform as the interest is just calculated on ending the trade.

As Oanda is a market maker, their data thus has less ticks than that of an ECN that may or may not be good depending what you want. The Oanda tick data is unique in its time span; it available since the start of 2004. Unfortunately, the available currency pairs are so limited down to 5 majors and tick data downloading the is restricted only to users that have an Oanda account with a minimum balance of $1000.

Being a market maker too, the volume isn't included in their tick data offered.

Available currency pairs:

EURUSD GBPUSD USDCAD USDCHF USDJPY

Downloading Tick Data

You should recall that as mentioned before; the tick data is completely free but you need to hold an Oanda account with a minimum of $1000, although there is an academic exemption.

Data downloading is queue-based. After going to the Oanda historical rates page, you then select "Make an order" that navigates to the tick data order form where you will be able to select the start and end dates, the currency pair and the output format; for the latter, everything should do beside XML. When ready, click "Place your order" that puts your request in the queue and once processing is finished that may take several hours to several days, you'll be emails. You can have only one request queued at a time. Also the data for the past two months cannot be requested, in other words the ending date in the request form mast not exceed the date two months ago.

When your file is ready, you can then download it.

Pros and cons

  • Pros:
  • Data since 2004 is available.
  • Easy to download.
  • Cons:
  • An account funded with a minimum of $1000 is mandatory.
  • Only 5 major pairs are available.
  • The whole process is queue-based and access to the file is not instant.
  • The most recent data cannot be requested, the tick data ending date is limited to two months prior to the request date.

MB Trading

Introduction

MB Trading is a US-based broker too, providing a wide range of products; concerning Forex, ECN services are offered to their clients. Metatrader 4 is available beside the Desktop Pro proprietary trading platform but with many peculiarities such as a 10k lot size, the requirement for a special authentication key, many partial fills, no expiration on orders and a daily server restart taking about 5 minutes at 5 PM US EST. Regardless of these things, they have very good spreads and decent commissions too.

The offered tick data is top-of-the-order-book with no volume, available for all majors and some minors pairs, their count almost compete the Dukascopy available pairs count. The data is totally free, but you may need to register for a demo account - for free - if you don't already have one. The oldest available date is Jan 2011.

Available currency pairs:

AUDCAD AUDCHF AUDJPY AUDNZD AUDUSD
CADCHF CADJPY CHFJPY EURAUD EURCAD
EURCHF EURGBP EURJPY EURNOK EURSEK
EURUSD GBPAUD GBPCAD GBPCHF GBPJPY
GBPUSD NZDCAD NZDJPY NZDUSD USDCAD
USDCHF USDDKK USDJPY USDNOK USDSEK

Downloading Tick Data

Go to the MB Trading tick history download page. Log in using your account credentials - just open a free demo account if you don't already have any. If you got strayed though registration, after logging in to your account, you can navigate to the page by clicking the Tick history link in the Tools menu.

Once reached the actual download page, you will have to select the currency pair you want then browse and enable every day you want to download, be certain that the days are consecutive and not randomly selected.

Afer selecting each and every day that you want is done, click "Download" and pray that it really works. It may run into a nasty problem that would stop downloading before it's completed and you may have to download each day individually.

After everything is done, you should get a file named Tick****.zip. Unpack it and inside you will find a pack of more zip files, one for each day. Unpack those too and when that's done, then concatenate in order them to be ready to be turned into an FXT - the same as in Pepperstone and Integral above.

Pros and cons

  • Pros:
  • Very good quality.
  • Wide group of available pairs.
  • Cons:
  • The start date of the available data is just 2011.
  • The data is split into days and the files must be unpacked and manually concatenated (merged).
  • Downloading the data is so complicated as you can maximally download 25 days at a time, making the whole process progressing very slow.
  • The pair name is included for each tick which is a complete waste of space and bandwidth.
  • The download process is quite unfriendly.
  • Concatenating the files for each day is hard to complete if you're doing it for a time span of several months.

HistData.com

Introduction

A group of traders are managing this project providing free tick data through it at their own expense a very thanked effort, there isn't any hidden interest at all.

The dates available are variable according to symbol and the oldest starts in 2000 for some major currencies, the data may lake some harmony, the CSV2FXT script will ask you to provide the GMT and DST offset manually.

As a unique bonus feature, text files are included in the archives to specify every gap larger than 60 seconds.

Available currency pairs:

AUDCAD AUDCHF AUDJPY AUDNZD AUDUSD
AUXAUD BCOUSD CADCHF CADJPY CHFJPY
ETXEUR EURAUD EURCAD EURCHF EURCZK
EURDKK EURGBP EURHUF EURJPY EURNOK
EURNZD EURPLN EURSEK EURTRY EURUSD
FRXEUR GBPAUD GBPCAD GBPCHF GBPJPY
GBPNZD GBPUSD GRXEUR HKXHKD JPXJPY
NSXUSD NZDCAD NZDCHF NZDJPY NZDUSD
SGDJPY SPXUSD TWIUSD UDXUSD UKXGBP
USDCAD USDCHF USDCZK USDDKK USDHKD
USDHUF USDJPY USDMXN USDNOK USDPLN
USDSEK USDSGD USDTRY USDZAR XAGUSD
XAUAUD XAUCHF XAUEUR XAUGBP XAUUSD
ZARJPY

Downloading Tick Data

Go to HistData.com and launch your download. Be certain that you got the Generic ASCII tick data. You can even use FTP download if you prefer that.

If you need to have more than one file in the same CVS, unpacking and concatenating is them in order the way to do that as mentioned before in Pepperstone and Integral.

Pros and cons

  • Pros:
  • A huge collection of available currency pairs.
  • Gap info.
  • Good date availability range for many currency pairs.
  • Cons:
  • The data source is inconsistent; 4 digit, fixed spread MM mixed with ECN.
  • Heavy download process, you can't download more than one month at a time, to do that you have to go back and select another month to download, however paid FTP access is available.
  • Inconsistent GMT offset and may be the DST too.

Gain Capital aka Forex.com

Introduction

Gain Capital which was founded in 1999, has some bad reputation as a market maker broker. In 2010 the NFA has awarded half a mil fine to them for some bad practices, as quoted “... engaged in abusive margin, liquidation and price slippage practices that benefited Gain to the detriment of its customers”. but their forex.com domain name must be bringing them huge profits.

Regardless of their vague trading conditions, a selection of tick data is available there, but with poor quality, not only is the large time spans missed, but those data may also be overlapping; sometimes you may get a previous week data in the current week. The only advantage of it is the abundance of available symbols; from currency pairs major, minor and exotic to metals and various stocks and indices.

Available currency pairs:

AUDCAD AUDCHF AUDJPY AUDNZD AUDUSD
AUXAUD* BCOUSD* CADCHF CADJPY CHFJPY
ETXEUR* EURAUD EURCAD EURCHF EURCZK
EURDKK EURGBP EURHUF EURJPY EURNOK
EURNZD EURPLN EURSEK EURTRY EURUSD
FRXEUR* GBPAUD GBPCAD GBPCHF GBPJPY
GBPNZD GBPUSD GRXEUR* HKXHKD* JPXJPY*
NSXUSD* NZDCAD NZDCHF NZDJPY NZDUSD
SGDJPY SPXUSD* UDXUSD* UKXGBP* USDCAD
USDCHF USDCZK USDDKK USDHKD USDHUF
USDJPY USDMXN USDNOK USDPLN USDSEK
USDSGD USDTRY USDZAR WTIUSD* XAGUSD
XAUAUD XAUCHF XAUEUR XAUGBP XAUUSD
ZARJPY

Downloading Tick Data

If you still insist to proceed, go to the Gain Capital rate data page and get the archives, it's better to use a download manager.

Some scripts for them are still available for download as a PHP script collection on the tick data downloads page in the Miscellaneous section.

  • There you will find:
  • A script to process the data into a CSV. This script is heavily based on the original by Carlos from the indo-investasi forum.
  • A script to somewhat filter & sort the resulted CSV.

It Shouldn't be too hard to figure out how they work, beside that these scripts are not supported in Birt's technical support.

Pros and cons

  • Pros:
  • A huge collection of available symbols including stocks and indices.
  • Cons:
  • Heavy download process.
  • The data is split into weeks and the files must be unpacked and manually concatenated.
  • Poor data quality, there are gaps and disruption.
  • No up to date tools are available there to process the data.
Last Modified On OCT 16 2013 By:
 
Information, charts or examples contained in this post are for illustration and educational purposes only. It should not be considered as an advice or endorsement to purchase or sell any security or financial instrument. We do not and cannot give any kind of financial advice. No employee or persons associated with us are registered or authorized to give financial advice. We do not trade on anyone's behalf, and we do not recommend any broker. On certain occasions, we have a material link to the product or service mentioned in the post. This may be in the form of compensation or remuneration.