AURORA Data Management Considerations and Best Practices

Begin with the End in Mind

To organize your input data more effectively, consider the process you intend to follow in producing your Aurora results and analysis. Answering the following questions:

Over what geographic scope will you be forecasting?
Over what time horizon will you be forecasting?
How frequently will you be updating your Aurora forecasts?

What inputs will you be bringing into Aurora?
What level of granularity will this data be in?
What format is this data in?
How frequently is this data refreshed/updated?

Change Sets

Change Sets are generally, the ideal/recommended approach for managing and tracking input data scenarios because they compare what is changed vs. what is in the underlying database.
- Keep Change Sets relatively small - you can merge at run time.This gives you the most flexibility and saves time debugging (if issues arise).
- For large changes (e.g., multiple 8760 demand profiles), consider making copies of input tables and switching between them (by using ‘in study’ check box) for performance reasons.

NOTE: Large Change Sets take time to merge/save. Ending the Aurora application or process via task manager during this process can corrupt Change Set data.

Organize/communicate responsibilities for changing input data to avoid overlaps if you plan to merge Change Sets from different users at a later date.
- Aurora can merge cell-by-cell two or more Change Sets with changes to the same record, but not the same record and column (cell).
Change Sets track adding/deleting of rows, but NOT columns. You cannot add columns with a Change Set active. Be careful of changing the table structure.

NOTE: If you add values to rows in a newly added column, Change Sets will track those, but if you switch to a previous database without that column, the Change Set cannot be applied. Aurora will warn you of a non-existing column and that changes to that column will be discarded.

Linked or Imported Tables from Excel

If you link or import an Excel worksheet, Aurora will look for certain columns in the table to recognize it as a specific Aurora table type (column names and data types). Otherwise, the table will appear as ‘unknown.’ An has column headers that do not fit the criteria for any other AURORA table types. Be sure to convert date formats to text (=text(A1, “m/d/yyyy”)) or data in those records will be ignored.
Be sure to Reload DB or force table load/refresh prior to running.
Generally, don’t use Primary Key column in external tables - let Aurora auto-assign.
Don’t mix linked tables with Change Sets, manage data separately in this case.

Save Edits to Database

Save Edits to DB makes sense when you have local, proprietary (better) information that you always want applied, not just in a single scenario, but usually only if there are many, many changes. Otherwise, keep change in a Change Set and treat as ‘base case’ in study cases list.

NOTE: Save Edits to DB eliminates the Change Set! If this feature is used, be sure to first copy the Change Set prior to using Save to DB in order to re-apply these Change Sets to future Energy Exemplar-released DBs. Also, do NOT activate this copied Change Set with the recently-saved (identical data) database. If this is done, the Change Set sees no difference between it and the underlying database, and will therefore be discarded. For this reason and others, it is also wise to store a backup copy of these 'copy Change Sets.' Use Change Set Files for storing backups.

Test Changes in Small Batches

When planning to make large sets of changes, it is wise to test them in small batches. Specifically, perform a short Aurora run after each batch to ensure data was entered properly and is flowing through the model as intended.
It is simple to set your period/hours to something very short and fast and direct output to a temporary database.
This practice alone can save significant time and effort in tracing troublesome input data.

Beware of Unseen Illegal Characters

When importing/copying data from other sources (especially from the Internet) into the AURORA database, beware of bringing in illegal characters. An especially troublesome one is the HTML code “&nbsp”, which is Unicode for non-breaking white space, because you can’t see it. If not removed, this can cause problems which are very difficult and time-consuming to diagnose.
Run the Energy Exemplar-provided utility (or another) to cleanup your database if you have any concerns.

Conventions for Numbering/Naming IDs

Don’t use sequential IDs for new records to the Resources (or other) tables that start from where Energy Exemplar-delivered IDs leave off or you will have duplicates when Energy Exemplar updates the database. Rather, use a company-specific prefix, for example MyCompany_111 to avoid potential conflicts and to preserve your additions through future Energy Exemplar updates.

Knowledge Base

Data Management Considerations