Network Interval Metering Data System (NIMDS)

Implementing a national solution to interval metering data management.

$ 0 M
0 mths
Photo Credit: APA Group



In 2004 APA implemented a PI-based system for managing gas consumption data for South Australian high demand customers in order to meet regulatory requirements for Full Retail Contestability (FRC). GTS partners Gavin Man and Andrew Todd were part of the original FRC project team establishing the PI system and the now expanded GTS team has been supporting it ever since.

APA had established a program of standardising SCADA systems across Australia and in 2014 were planning an upgrade to their QLD networks SCADA system.  Rather than rebuild the QLD FRC components  as part of the SCADA upgrade (and maintain two similar systems), it was decided to create an enterprise-wide FRC solution that could consolidate the needs of all states into a single application.

Technical Situation

GTS had completed the development of a national PI System for APA in June 2012 as part of the Metering Data Management (MDM) project which was already integrated into APA’s SCADA systems.

The system comprised a PI archive collective, Asset Framework (AF) and Advanced Calculation Engine (ACE).

The original FRC system was still functioning and most of the business requirements were unchanged however the more recent MDM system was built on a newer PI platform and offered many improvements over the legacy FRC environment.

It was decided that the NIMDS project would be built on the national PI System platform modelling functionality on the legacy SA FRC system including:

  • Data capture from SCADA sources (through PI), from files (in multiple formats) provided by FTP, email or manual entry;
  • Implementation of a complex network model for all site configurations including multiple meters or runs, different data types and pressure correction factors (PCF) etc;
  • Processing of SCADA data to derive hourly and daily volume and energy based on site configuration (e.g. calculation from index values using PCF);
  • Automatic estimation of missing values according to industry rules;
  • Implementation of complex, user-configurable validation rules to be applied in real-time;
  • Scheduled report generation and delivery to internal and external stakeholders;
  • Data processing, validation and delivery within timeframes defined by the gas market regulator;
  • Implementation of a retrospective monthly reconciliation process on receipt of manual meter read data;
  • User interface allowing users to review, revise and approve data; and
  • Design and build the dashboard model and presentation layer


A Gas Distribution Network model was developed in OSIsoft’s PI Asset Framework (AF). PI AF is a powerful data infrastructure tool because it allows a logical, hierarchical structure to be defined that combines data from any source (such as PI points interfaced to SCADA, databases, or other systems) with calculations or static data.

This is a great way to abstract any complexity in deriving data from the users. For example a user looking for the gas volume through a gate station can simply navigate to the gate station in the model and look at its volume attribute. Behind the scenes there may be multiple data points and complex calculations but all of that is hidden from the user.  

This is a great way to abstract any complexity in deriving data from the users. For example, a user looking for the gas volume through a gate station can simply navigate to the gate station in the model and look at its volume attribute. Behind the scenes there may be multiple data points and complex calculations but all of that is hidden from the user.

The NIMDS AF model included all of the relevant SCADA-based PI points as well as relatively static network data such as gas retailer information and Meter Installation Registration Number (MIRN); and processing instructions (i.e. how the PI ACE modules would process data from each site).

The main use of the PI AF model would be to provide what was essentially a large and complex configuration file for processing scripts developed in OSIsoft’s PI Advanced Calculation Engine (ACE).

PI ACE was used to create data processing scripts that extracted the PI AF model configuration, tapped directly into the PI Event Pipe to process data in real-time using the rules supplied by the PI AF model, apply data selection and estimation algorithms and output the results to PI points.

Additionally,(and also in common with the MDM application) the data produced needed to be pre-validated according to both regulatory requirements and business processes so that possible issues could be detected and flagged for human review. A set of validation rules and associated parameters was defined by the business, configured in PI AF, and scripted in PI ACE modules. After the incoming data was processed, it then passed through a validation process with a validation record point used to record the history of test results.

The NIMDS solution was required to facilitate a number of day-to-day business functions via a web-based interface including:

  • Maintaining certain system configuration settings
  • Reviewing and authorizing data to be released to the market
  • Resolving any data validation issues
  • Manual correction of calculated values
  • Monthly reconciliation processes involving loading manual meter read data and reconciling against telemetered data received

An ASP. NET web site was created to serve the user interface needs and combined with a data access component responsible for interfacing to the AF and PI database to fetch and return datasets to the web site.

In addition to user interface screens for day to day processes, there were dozens of scheduled or ad hoc reports required. While some legacy reports could be replaced by auto validation, many more were required for a number of purposes, from simple end of day or end of month gas consumption reports, to troubleshooting data sets (for example when queries were received from customers), right through to data process status reports and reconciliation reports.

Some of these were used internally while others needed to be provided to customers, retailers or other external parties.

The solution design included a number of different approaches to best suit the different requirements:

  • Scheduled reports could be automatically generated, validated and delivered without any human effort
  • Some reports were intended to be displayed for review (much like a manager’s dashboard) and these were integrated into the user interface to be generated on demand

Others were more dynamic and content varied widely according to a user’s needs; rather than developing a fixed report for all scenarios, OSIsoft’s Coresight was integrated into the user interface to provide users with a simple but powerful tool to search or browse data sources and easily create custom displays to suit their own needs.

The outcome was a powerful, reliable and scalable enterprise application providing for the replacement of the legacy Queensland FRC system but also for the eventual migration of similar systems in other states.

Project Outcomes & Benefits

This system provided the following benefits:

  • Reduced application maintenance effort and license costs with multiple systems consolidated into a single platform;
  • Less reliance on third party support with fewer applications involved;
  • Data that was previously unavailable or difficult to source is easily accessible across the enterprise;
  • Labour intensive manual validation processes were replaced by automated processes;
  • Quality was improved with a consistent business process applied to data from all states; 
  • Duplication of effort was removed by centralising data processing to a single national team;
  • A National Interval Metering solution;
  • Standardising and simplifying complex processes; and
  • Improving reliability and quality of data.
Andrew Todd
Andrew Todd

Chief Executive Officer

Multiple systems were consolidated into an enterprise platform reducing software maintenance costs while improving data quality and reliability.

Andrew Todd

Chief Executive Officer at GTS Group

Other case studies