Norgate Data Updater Tools
The historical data for any security can be viewed in table form via the Data Viewer. Click on Tools > Data Viewer and then
select a Database or a Watchlist. You can search for a security by symbol or name. Once you have highlighted a security, the data will appear in table form, with data fields/columns that are appropriate to the security type.
- After clicking the Data Viewer icon, wait a second or two for the module to open. Clicking the icon again will open a second instance of it.
- The historical data for a stock may show different values depending on how you have chosen to have it adjusted under "Data Configuration > Stocks". Also, the data for any security may show differently depending on the selection for "Padding".
- You can use "Data Viewer" as a cross-check if viewing the data in another application suggests that a symbol might be missing from the database, or that some data might be missing or incorrect.
Export Task Manager
Use the Export Task Manager to manage the export of historical price data in txt/csv format. The first step is to create an "Export Task".
Creating a Task
Open the Export Task Manager and click the "Create" button to launch the Norgate Data Export Wizard which will step you through the process of constructing a task.
Step 1 - Select what to export
You can elect to export the contents of a single Database or Watchlist or add mutiple items of either sort to the same task.
Step 2 - Configure the underlying data
Price & Volume Adjustment is only relevant if you are exporting data for Equities.
Step 3 - Select the fields you want to include
Some fields are relevant for all security types whilst others are specific. For instance -
"Volume" is not relevant for Economic or Cash Commodities data.
"Delivery Month" is only relevant for Continuous Futures.
If you are exporting a variety of security types in the same task, you can tick every field that is needed to cover the full variety. The output will only include an irrelevant field for a particular security type if you tick the option to "Output empty fields".
Some applications may expect the data to contain a field for "Time" as well as "Date". If so, tick the field "Time (Dummy Value)". You can enter a dummy value at the next step.
Step 4 - Field Options
This is where you define the form of the output. Click the "Show Example" button to preview the output. You can click the button repeatedly to show a series of examples.
Some selections may produce ambiguous output. For instance, you may have earlier selected "Security Name" as a field to be output. Security names will contain spaces. If you then select "SPACE" as the Field Delimiter, a space in the Security Name will be interpreted as a field break. To deal with such conflicts, tick the option to "Wrap fields in Quotes". This works to contain the data in the relevant field.
Step 5 - Date Range Options
You can elect to export all of the data for each security or only part of it.
If you want to have a fixed "start date" for the export with the output always up to "today", select the "Between date range" option. After specifying the desired "start date", set the "last date" to a distant date in the future.
Step 6 - Folder and File Options
This is where you specify the location for the exported data.
The Destination Folder holds the data in sub-folders that are named according to the items that have been exported (an item is a Database or Watchlist).
The option to "Refresh Existing Sub-folders" is an important one. If this option is left unticked, securities that no longer exist will remain in place from previous exports. Ticking the option serves to clear all sub-folders first which allows a contemporary (fresh) export each time. We strongly recommend that export sub-folders be kept free of data files from other sources, or any other files at all. If the option to "Refresh Existing Sub-folders" is ticked, they will be wiped away!
The option to "Export a names.txt file" enables the creation of a text file in each export sub-folder to show the Symbol and Name of each security in the sub-folder.
"File extension" is user-determined. The default is "csv" (comma separated values) to match the default Field Delimiter. ASCII files delimited in other ways typically have the extension "txt".
If you assign a name to the task, it will be saved for future use. The task can be run "on finish" whether saved with a name or not.
Running a Task
To run an Export task, open the Export Task Manager, tick the relevant task (or tick several tasks) and then click "Run".
You can also set NDU to run tasks automatically after a data update via the tick-box options under Settings > Update Mode.
If you have created a number of tasks, but only want some of them to run automatically, you can change the "State" of the others from "Active" to "Paused" via the Pause/Unpause command.
To run active export tasks after an update launched via NDU.Trigger,
add the command line parameter
Editing a Task
You can change any aspect of an existing Export Task via the Edit command. For instance, if you would like to export an additional item, you can add it to an existing task rather than create a new task for it.
Things to watch out for
- Make sure that all files previously exported are closed before running an Export. If a file is found open, the export should continue, but the open file will be skipped.
- Make sure that the Destination Folder continues to have "write permission" and is otherwise accessible.
- Don't add files from elsewhere to your Destination sub-folders.
- Don't rename a Watchlist that's currently part of an Export Task. Instead, first remove the Watchlist from the task via Edit or simply delete the task. Then rename the Watchlist and add it to Export again.