Massive/Polygon Backfill Tool by MabeKit

Documentation for the Massive/Polygon Backfill Tool by MabeKit for downloading historical bar data from Massive/Polygon and importing it into AmiBroker.

This tool downloads market data from a Massive/Polygon subscription using the Minute Bar Aggregate files.

It includes delisted symbols and automatically adjusts for splits.

Installation

Extract MassiveFlatBackfill-dist.zip to a directory on your machine. (I chose e:\amidata\backfilltest)

Step 1: Download the Data

Practice the entire process on a small set of recent days, since doing the whole backfill takes several hours. Follow the steps below to test.

Look at the README.md file in the extracted directory for the full list of options.

Open Windows PowerShell, navigate to the directory where you extracted the tool, and run:

cd e:\amidata\marketdatatest
.\massiveflatbackfill.exe --start-date YYYYMMDD --data-dir e:\amidata\marketdatatest\backfilltest

(Replace YYYYMMDD with a recent date, e.g. 20260515 for May 15, 2026.)

Remember, of course, that your local paths will differ.

This downloads 1-minute bars from the specified date for all symbols that traded.

After it completes, you will see per-symbol CSV files in the data directory:

Step 2: Adjust for Splits

Here's the command for adjusting the downloaded data for splits:

.\massiveflatbackfill.exe --adjust --data-dir e:\amidata\marketdatatest\backfilltest

Step 3: Create the Import File

Now, create the single import file. In Windows PowerShell, run:

.\massiveflatbackfill.exe --import --data-dir e:\amidata\marketdatatest\backfilltest

This will create a file called .import.qcolcsv in the current directory:

Step 4: Import the File Into Amibroker

There are some simple steps in AmiBroker to make this process seamless.

There are two files in the AmibrokerFormats directory in the zip file:

• mabekit_intraday.format

• mabekit_daily.format

Copy these files to the Formats directory in your AmiBroker installation. Most likely the path is: C:\Program Files\AmiBroker\Formats.

Then open the import.types file in that same Formats directory.

Add these two lines at the bottom:

MabeKit Intraday Import (*.qcolcsv)|*.qcolcsv|mabekit_intraday.format
MabeKit Daily Import (*.qcolcsv)|*.qcolcsv|mabekit_daily.format

Once this is set up properly, when you go to File -> Import ASCII..., you should see the MabeKit Intraday Import option:

Now you should be able to go to AmiBroker and import the file.

Choose File, Import ASCII...

Navigate to the folder where the import file was created.

Choose the MabeKit Intraday Format item from the Files of type field.

Choose .import.qcolcsv from the list and click Open.

Open a chart for one of your test symbols. You should see 1-minute bars starting from your start date.

Full Backfill

Once the test works, run the full backfill. In Windows PowerShell, run:

.\massiveflatbackfill.exe --start-date 20160101 --data-dir e:\amidata\marketdatatest\backfilltest

This will take several minutes for the full symbol list.

After it completes, create the import file. In Windows PowerShell, run:

.\massiveflatbackfill.exe --import --data-dir e:\amidata\marketdatatest\backfilltest

Automating the Entire Process

In the AmibrokerFormats directory in the zip file, there's a file called import_to_amibroker.py.

It can be used to automate the entire process.

Here's a batch file I use to update my backtesting database daily:

.\massiveflatbackfill.exe --barsize 1 --data-dir .\flatfiletest02\

.\massiveflatbackfill.exe --adjust --data-dir .\flatfiletest02\

.\massiveflatbackfill.exe --import --data-dir .\flatfiletest02\

python import_to_amibroker.py --import-file .import.flat.qcolcsv --definition e:\amidata\AmibrokerFormats\mabekit_intraday.format --database e:\amidata\localdatabase01

The entire process can be run from a Scheduled Task.

The initial data import will take hours to complete, but after that, each incremental update will be very fast.

It takes ~2 minutes for the entire process to run to update a single day.

Troubleshooting

"Invalid date format" error

Make sure you selected MabeKit Intraday Format from the Files of type dropdown when importing — not the default format.

No data on chart after import

Check that your database base time interval is 1-Minute and your chart date range covers the imported dates. Check Window, Output for the import log.

Format not showing in import dialog

Make sure the format file is in the Formats folder and restart AmiBroker after editing import.types.