Norgate Data Updater FAQ

Installation

If I install NDU under a supported but older version of Windows, will it work just as well?
Can I install NDU on a Mac?
I've stopped updating from one of my two computers, so why am I prevented from installing on another?
I have installed the NDU database on d: drive. Why am I getting a pop-up warning about free disk space on c: drive?

Database

How can I find my Norgate Data database?
How do I change the Database Location?
Can I search the Database?
Do I need to keep a backup copy of the Database?
Do I need to keep a backup copy of my Watchlists and/or Export Tasks?
The Database tab shows that I have a database marked as "Corrupt". How do I fix it?
The Database tab shows that all my databases are marked as "Corrupt". How do I fix this?
I am getting repeated instances of database corruption. What could be causing that?
Is the NDU database static with regard to historical data?
When it comes to Price & Volume Adjustment for stocks, what events count as "Capital reconstructions" and "Special Distributions"?

Usage

How can I improve the update and data access speed?
NDU started to download an historical database and then stopped. What could the problem be?

WatchList Library

Where are my Watchlist Library files stored?
How do I transfer the NDU Watchlists created on one machine to the Norgate Data database on another machine?
My Watchlist Library includes watchlists for "Current & Past" index constituents. To restrict a back-test to historical index constituents, do I simply reference such a watchlist in a back-test?
When a symbol in a watchlist for "Current & Past" has a date suffix - like XYZ-201506 - does the suffix indicate when the stock was removed from the index?
Why can't I filter US delisted stocks by exchange?
Why does the S&P 500 Watch List contain more than 500 tickers?

Installation

If I install NDU under a supported but older version of Windows, will it work just as well?

It should. But you need to make sure that your version of Windows has been activated and kept up-to-date with critical updates. In particular, NDU can work under Windows 8.1 but not under Windows 8.

Can I install NDU on a Mac?

Not directly. NDU will only work under Windows. However, you can run NDU indirectly on a Mac via Windows virtualization software ("Windows Emulators") such as Parallels Desktop, VMWare, VirtualBox etc.

I've stopped updating from one of my two computers, so why am I prevented from installing on another?

We don't track individual machines - instead we keep count of the total number of machines that have logged in. If you want to abandon one of two existing installations, and install on a new machine, the process is to notify us so that we can re-start the count. Ask us to "reset my hardware keys" or "restart my machine count". As soon as 2 machines have logged in again, the limit has been reached again (we don't care which ones they are).

I have installed the NDU database on d: drive. Why am I getting a pop-up warning about free disk space on c: drive?

"Temp files" are typically directed to a Windows folder on c: drive by default.
After NDU has been installed, it requires a minimum of 500 MBs of free disk space on c: drive to keep functioning.
We provide a facility to give "advance warning" about current space on c: drive (Settings > Notifications). You can either untick the facility or set the threshold to suit yourself.

Database

How can I find my Norgate Data database?

You can find the location of your database by opening NDU and clicking on the Database tab or on Settings > Database Location. To open the folder, go Settings > Database Location and click "View Database folder contents".
Please note that the default database folder "c:\ProgramData\Norgate Data" might be hidden by Windows and therefore not shown in Windows Explorer.

How do I change the Database Location?

See the instructions provided here.

Can I search the Database?

Yes. The Data Viewer allows you to search by symbol or name. To do a refined search with filtering, use the Watchlist Library interface.

Do I need to keep a backup copy of the Database?

No. If any component database goes missing or becomes corrupt, you can click on the Database tab and use the Download command to fetch a replacement.

Do I need to keep a backup copy of my Watchlists and/or Export Tasks?

Not in particular. NDU automatically creates backups for you in case you want to restore a previous version. These backups are kept in your Database folder under the "Configuration" sub-folder. For example -
C:\ProgramData\Norgate Data\Configuration\Watchlists\Backups
But the backups that NDU creates won't help in the case of something like a disk failure, for which only an off-site backup can provide recovery.

The Database tab shows that I have a database marked as "Corrupt". How do I fix it?

Select the corrupt database and click the Download command.

The Database tab shows that all my databases are marked as "Corrupt". How do I fix this?

First of all, refer to the FAQ above. The instructions below apply only if the Download command is not working to replace Corrupt databases.

Click on Settings > Database Location > View Database folder contents and then close NDU.
Find the file core.ngdb in the database folder and delete it.
Then open NDU again and do a data update (which should deliver a replacement).

I am getting repeated instances of database corruption. What could be causing that?

The most likely explanation is that your Security System/Anti-Virus software is performing real-time anti-virus scanning on database files at the same time that NDU is trying to write information to them. In that case, the solution would be to configure the software to exclude the NDU database folder from real-time anti-virus scanning.

Is the NDU database static with regard to historical data?

No. Corrections come through to us from our primary upstream suppliers and these are transmitted automatically. Other errors and omissions are corrected by us as soon as they are discovered.

For stock market data, corrections may apply to any number of things, including:

• Trade corrections/cancellations
• Index prices and volumes
• Listing status (listing venue, trade suspension and delisting)
• Dividend type
• Dividend ex-date
• Sector/Industry classification
• Surviving entity determination (mergers & acquisitions)
  And so on ...
  When it comes to Price & Volume Adjustment for stocks, what events count as "Capital reconstructions" and "Special Distributions"?
  

  Capital reconstructions include splits & consolidations (reverse splits), bonus issues (stock dividends) and rights issues.
  Special distributions include capital returns, special dividends and distributions associated with spin-offs.

  Adjustment factor calculation
  Hide

  A capital reconstruction involves a change to the number of shares on issue. The price adjustment factor for splits and bonus issues is calculated as (Previous number of shares / New number of shares). The easiest event to understand is a "share split" - for instance, a 2 for 1 split where the number of shares on issue is doubled. In this case, all historical prices before the ex-date of the split are multiplied by 0.5 in order to bring them into line with post-split prices. At the same time, historical volumes are multiplied by 2.
  
  Capital returns, special dividends/distributions and ordinary dividends are all adjusted for in the same way, as in the generic example below:
        Closing Price = $100
        Distribution = $10
        Expected Opening Price = Closing Price - Distribution = $90
        Adjustment Factor = $90/$100 = 0.9
        All prices before the ex-date of the distribution are multiplied by 0.9

  
  

  Usage

  How can I improve the update and data access speed?
  
  1. Exclude your NDU database folder from real-time antivirus scans. To find the location of your NDU database see How can I find my Norgate Data database?
  2. Store your database on an SSD drive (you can either replace your main system drive or purchase an SSD as an additional drive). Solid State Drives increase performance significantly.

  Some charting/analysis applications also have their own database folder. See the relevant Plugin FAQ for more information on speeding up data access:
  AmiBroker
  RightEdge

  NDU started to download an historical database and then stopped. What could the problem be?
  

  This is typically due to a low quality internet connection failing to support a multithreaded download. Go to NDU > Settings > Connection and select Database Download Method = Single to see if that helps.

  WatchList Library

  Where are my Watchlist Library files stored?
  

  The files are stored in binary format in the \Configuration\Watchlists folder under the Norgate Data database folder. They are additionally stored as text files in the \Configuration\Watchlists\txt sub-folder.

  Note: If you have an older NDU database installation, the Watchlists folder may be located directly under the Norgate Data database folder and not under the Configuration folder.

  How do I transfer the NDU Watchlists created on one machine to the Norgate Data database on another machine?
  

  Follow the instructions for "How can I find my Norgate Data database?" to open the NDU database folder. Copy the entire Watchlists folder from the first machine to replace the folder on the second. Once done, run an NDU update. AmiBroker users should then also run database maintenance.

  Note: If the second machine has a "Configuration" folder, make sure that the Watchlists folder is pasted there as a sub-folder (see the FAQ and Note above regarding the "Configuration" folder).

  My Watchlist Library includes watchlists for "Current & Past" index constituents. To restrict a back-test to historical index constituents, do I simply reference such a watchlist in a back-test?
  

  No. A watchlist is just a simple collection of stocks. A watchlist for "Current & Past" collects all the stocks that were ever members of an index. But the watchlist itself contains no information about when any stock was actually in the index. To gain that information, you need to reference the NorgateIndexConstituentTimeSeries plug-in function. Given such a reference, you can run the back-test efficiently over the watchlist instead of running it inefficiently over all stocks.

  When a symbol in a watchlist for "Current & Past" has a date suffix - like XYZ-201506 - does the suffix indicate when the stock was removed from the index?
  

  No. The date suffix distinguishes delisted stocks from currently-listed, with the suffix indicating the year and month that the stock was last traded. Past members of an index can still be currently-listed.

  Why can't I filter US delisted stocks by exchange?
  

  This is because US delisted stocks, in general, cannot be associated with a particular exchange. A stock may change its listing venue (move from one exchange to another). A stock may also lose its listed status and trade OTC (over the counter) for a period of time before being re-listed. It is not uncommon for a delisted stock to have been associated with 3 different venues over its lifetime. The "last known venue" is not necessarily useful because a stock may have been listed on a major exchange for most of its lifetime before finishing up OTC.

  Why does the S&P 500 Watch List contain more than 500 tickers?
  

  In 2014, separate share class lines were added for Discovery Communications and Google Inc to bring the total number of constituents to 502. Further such additions brought the total to 505 after the September 2015 re-balance. However, in July 2017, S&P Dow Jones Indices announced that it would no longer add companies with multiple share class structures to the S&P Composite 1500 or its component indices (the existing constituents were excused from the methodology change).