Norgate Data Norgate Data
 

AmiBroker Usage

Database Settings

Database settings allow the user to configure the data, with options provided for:
Date Padding Date Padding options
Price AdjustmentPrice Adjustment options
To check the existing settings, or to change them, click File > Database settings > Configure.

Database Maintenance & Automation

Database Maintenance
Automating Database Maintenance
Automating the Database Plugin Settings in Batch Mode

Indicators & Functions

Important Note
Making use of Historical Index Constituents
Making use of Historical Index Constituents during ranking
Determining when a stock was listed on a major exchange
Determining when a stock was a Blank Check Company/SPAC
Determining whether a date has been padded
Referencing the Original Unadjusted Close and Volume
Accessing Dividends
Accessing Dividend Yield
Determining whether a capital event has occurred
Accessing Fundamentals
Accessing Sector/Industry Classification
Determining the Sector/Industry Index symbol for the current security
Business and Financial Summary


Database Maintenance & Automation

Database Maintenance

Database maintenance serves to:

When the Norgate Data plugin detects that any maintenance is required, you will see the message "MAINT" at the bottom right on the AmiBroker taskbar in the plugin status box -


If the MAINT message is shown, RIGHT CLICK on the plugin status box. A window will pop up, showing the progress of the maintenance routine.



Once the routine has completed, the box colour will turn to green and the plugin status box will change to "OK".



Database Maintenance can be run at any time, whether the message box shows MAINT or not, by right-clicking the plugin status box. For example, if you accidentally delete a symbol from the AmiBroker database, then running Database Maintenance will restore it.

If you create or modify a Watchlist in NDU's Watchlist Library, Database Maintenance must be run so that AmiBroker can pick up the new list or refresh the existing list.

Automating Database Maintenance

In AmiBroker v6.20 and above, you can use the Batch Processor feature (File > New > Batch > Insert) to:

  1. call the NDU Trigger utility to run a background data update in NDU, and/or
  2. run the Database Maintenance
The batch settings for a "default" NDU installation would be as follows (if you installed NDU to a non-standard directory you will need to adjust as required):

ActionParameter
Execute and Wait
c:\Program Files\Norgate Data Updater\bin\NDU.Trigger.exe UPDATE CLOSE WAIT
Data Plugin Command
NorgateDatabaseMaintenance

After adding the batch commands, it would look like this:



If desired, you can then use AmiBroker's Scheduler (Tools > Scheduler), to run this Batch at your preferred time(s).

If you only want the Database Maintenance to run when the MAINT message is shown, use the following Data Plugin Command instead:

NorgateDatabaseMaintenanceIfRequired

Automating the Database Plugin Settings in Batch Mode

The NorgateData plugin settings for Date Padding and Price & Volume Adjustment can be configured via the Batch Processor feature in AmiBroker v6.20 and above, prior to running any scans or backtests by using the following Data Plugin commands:

NorgateSetPadding=None
NorgateSetPadding=MarketDays
NorgateSetPadding=AllWeekDays
NorgateSetPadding=CalendarDays
NorgateSetPriceAndVolumeAdjustment=None
NorgateSetPriceAndVolumeAdjustment=CapitalReconstructions
NorgateSetPriceAndVolumeAdjustment=CapitalReconstructionsAndSpecialDistributions
NorgateSetPriceAndVolumeAdjustment=All




Indicators & Functions

Important Note

To use any of the Norgate-provided functions, you will need to add the following line to the top of any AFL file you use for scanning/backtesting/indicator plots.

#include_once "Formulas\Norgate Data\Norgate Data Functions.afl"
Show full list of Indicators/Functions

Making use of Historical Index Constituents

To determine whether a stock was in a particular index on a given date, use the function NorgateIndexConstituentTimeSeries by adding it to your "Buy" condition, for example:

AND NorgateIndexConstituentTimeSeries("index symbol or index name")
where the index symbol is $DJI (Dow Jones Industrial Average), $RUA (Russell 3000) and so on.

The index symbol or index name must be wrapped in "quotes". For example:

AND NorgateIndexConstituentTimeSeries("S&P 500")

If you wish to stipulate membership of one index but not of another, use (for example) -

AND NorgateIndexConstituentTimeSeries("$XAO") AND NOT NorgateIndexConstituentTimeSeries("$XTO")

Note 1: The function does not return dates. Instead, it returns true or false (1 or 0) for index membership for any date.
Note 2: You will need either a Platinum or Diamond level subscription to access historical index constituent information. At Silver or Gold level, the function will always return 0.

When using the NorgateIndexConstituentTimeSeries function to run a back-test against a particular index, we recommend referencing the Watch List that contains the current and past constituents for that index (specify this Watch List in the Analysis Filter window). This will provide a faster result than running the test against all symbols. For further information on this topic, see here.

An indicator plot is also provided in the Chart pane under the Norgate Data section.

Making use of Historical Index Constituents during ranking

A further advanced function exists that allows you to reference an alternative symbol rather than using the current symbol. This would normally be used in ranking mechanisms that use StaticVars in conjunction with SetForeign.

NorgateIndexConstituentTimeSeriesOther("index symbol or name","symbol")

This function will return a 1 (ie: "True") if the security was in the specified index on a given day or a 0 if False.

Determining when a US stock was listed on a major exchange

Some US stocks have had periods where they were not listed on a major exchange and traded via OTC/Pink Sheets.

In backtesting, you can elect to only trigger a trade if the stock was listed on a major exchange (NYSE, Nasdaq, NYSE American, NYSE Arca, Cboe BZX, IEX) on the signal date by adding the function NorgateMajorExchangeListedTimeSeries to your "Buy" condition:

AND NorgateMajorExchangeListedTimeSeries()

Note 1: The function does not return dates. Instead, it returns true or false (1 or 0) for major exchange listing for any date.
Note 2: You will need either a Platinum or Diamond level subscription to access major exchange listing information. At Silver or Gold level, the function will always return 0.

You could also incorporate this indicator in code used to plot a histogram, ribbon, or to modify candle/bar colours to signify major exchange vs OTC trading periods.

Determining when a US stock was a Blank Check Company/SPAC

Some US stocks have had periods where they were designated as Blank Check Companies - this includes SPACS (Special Purpose Acquisition Companies).

In backtesting, you can elect to only trigger a trade if the stock was not a blank check company at the time by adding the function NorgateBlankCheckCompanyTimeSeries to your "Buy" condition:

AND NorgateBlankCheckCompanyTimeSeries() == 0

Note 1: The function does not return dates. Instead, it returns true or false (1 or 0) indicating whether the stock was a Blank Check Company for any date.
Note 2: You will need either a Platinum or Diamond level subscription to access Blank Check Company information. At Silver or Gold level, the function will always return 0.

You could also incorporate this indicator in code used to plot a histogram, ribbon, or to modify candle/bar colours to signify such periods.

Here is an example of how you could filter out Blank Check Companies; as well as any stocks that were formerly Blank Check Companies which haven't completed a "seasoning" period of 6-months (125 trading days):

blankcheckcompany = NorgateBlankCheckCompanyTimeSeries();
bstransition = BarsSince(blankcheckcompany > 0);
nonblankcheckcompanyorseasoned = blankcheckcompany == 0 AND (IsEmpty(bstransition) OR bstransition > 125);
Buy = <your entry rules here> AND nonblankcheckcompanyorseasoned;

Determining whether a date has been padded

If you have the database configured to allow Date Padding, then you may wish to use the function NorgatePaddingStatusTimeSeries in your backtest to determine whether the data on a specific date has been padded or not (so as to avoid entering or exiting a position on a date where the stock did not actually trade).

You can do this by incorporating the following into your code:

paddingstatus = NorgatePaddingStatusTimeSeries();

Referencing the Original Unadjusted Close and Volume

Depending on your plugin configuration settings for Price & Volume Adjustment, the price and volume data may be adjusted to account for the effect of capital events and dividends.

If your trading system needs to reference the original unadjusted closing price of a security, use the function shown below.

oc = NorgateOriginalCloseTimeSeries();

Likewise, if you need to reference the unadjusted volume, use:

ov = NorgateOriginalVolumeTimeSeries();

Accessing Dividends

The Aux2 field shows any dividends that have not been accounted for in the price data by the "Price Adjustment" method. If all types of dividends and distributions have been accounted for, and you still wish to view a dividend entitlement for a particular day, you can use the following function to determine whether there was a dividend entitlement on that day:

NorgateAllDividendsTimeSeries(dividend type)
Where Dividend Type can be:

If there are multiple dividends on any given day, they will be summed.

This indicator may be useful for detecting dividend timing or simply as a visual indicator on a chart to show that a dividend entitlement occurred on a particular day.

Accessing Dividend Yield

To access the dividend yield, incorporate the following into your code:

dividendyield = NorgateDividendYieldTimeSeries();
An indicator plot is also provided in the Chart pane under the Norgate Data section.

Determining whether a capital event has occurred

This indicator shows when a capital event occurred. Effective on holding the security at the close on the day prior to the ex-date. Events include splits, reverse splits, bonus issues/stock dividends (dividends paid as stock), renounceable (rights) and non-renounceable issues of shares, and complex reorganizations of capital.
This function will return a 1 (ie: "True") if a capital event occurred on a given day or a 0 if False.
To access the capital event function, incorporate the following into your code:

capevent = NorgateCapitalEventTimeSeries();
An indicator plot is also provided in the Chart pane under the Norgate Data section.

Accessing Fundamentals

AmiBroker has a number of in-built data fields related to fundamentals, as described here which are populated by Norgate and can be accessed by the in-built GetFnData function.

Norgate also provides a wider range of fundamental fields (some companies have up to 140 fundamental fields). To access this data, call the following function

NorgateFundamentals(fundamental_field)

For example, to determine the most recently reported trailing twelve month payout ratio you would incorporate the following into your formula:

ttmpr = NorgateFundamentals("ttmpayrat");

Fundamental data is often released after the fact ("delayed") and does not always relate to the date of the current price bar. More information on this topic can be found here.

If a particular field is not available for a security, a Null value is returned. You can use the IsNull() function to determine this condition.

In order to determine the date that the fundamental data applies to, you can use the following functions:

For example, to see the date to which the trailing twelve month payout ratio relates to, use:

ttmprdate = NorgateFundamentalsDate("ttmpayrat");

Accessing Sector/Industry Classification

Functions in this area can be used to reveal the classification ID, the classification name or the classification description for a given stock under a supported classification scheme.

Please note: Not all stocks are necessarily classified.

Supported classification schemes are -

The following examples refer to TRBC:

trbccode = NorgateClassification("TRBC");
trbc = NorgateClassificationName("TRBC");
trbcdescription = NorgateClassificationDescription("TRBC");

For Nasdaq:AAPL this would return -

To obtain a classification at a different level, you can use NorgateClassificationAtLevel, NorgateClassificationNameAtLevel and NorgateClassificationDescriptionAtLevel. Detailed descriptions are typically only available at the lowest level of the classification. For example, to obtain the 2nd level of the TRBC classification you would use:

trbccodeAtLevel2 = NorgateClassificationAtLevel("TRBC",2);
trbcAtLevel2 = NorgateClassificationNameAtLevel("TRBC",2);
trbcdescriptionAtLevel2 = NorgateClassificationDescriptionAtLevel("TRBC",2);

For Nasdaq:AAPL this would return -

Determining the Sector/Industry Index symbol for the current security

To retrieve the industry index symbol for the current security, you can use the NorgateIndustryIndex function as follows:

indexsymbol = NorgateIndustryIndex("<index family>", <level> ,"<index type>");
where:
<index family> can be the symbols: $SPX, $SP1500, $XJO or $XKO. You can also use the longer names of S&P 500, S&P 1500, S&P/ASX 200 or S&P/ASX 300 for readability
<level> is an integer from 1 to 4
<indextype> is either PR (price return) or TR (total return)

For example, to determine the S&P 1500 Level 4 (Sub-Industry) price return index applicable to the current symbol, you would use:
indexsymbol = NorgateIndustryIndex("$SP1500",4,"PR");

Please note: indexsymbol will return "n/a" if:

  1. the current security is not a member of the specified index family
  2. the current security is not classified
  3. the relevant index does not currently exist at the given level due to a lack of eligible constituents.

To circumvent condition (1), you can determine a “corresponding” Industry Index symbol. For instance, the security of interest may not be a member of the S&P 1500 (and therefore not a member of the S&P 500), but you may wish to track it alongside an S&P 1500 or 500 Sector/Industry Index nevertheless. To accomplish this, use the NorgateIndustryIndexCorresponding function in place of the NorgateIndustryIndex function described above. For example:

indexsymbolcorresponding = NorgateIndustryIndexCorresponding("$SP1500",4,"PR");

Business and Financial Summary

There are several functions related to Business and Financial summaries of companies such as NorgateBusinessSummary(), NorgateBusinessSummaryLastUpdate(), NorgateFinancialSummary(), NorgateClassificationName(<classification scheme>) and NorgateClassificationDescription(<classificationscheme>).

One useful place to display this information is AmiBroker's Interpretation window (the Interpretation window appears if it is ticked under the AmiBroker > Window menu).

Norgate has pre-built a formula to add these business summaries to the Interpretation window. To incorporate this formula file, right click anywhere on a price chart, select "Edit formula .." and then copy/paste the following code so as to append it after the formula:

#include_once "Formulas\Norgate Data\Business and Financial Summary.afl"

For example, here is the Business and Financial Summary for NYSE:GE