The Blog of operational excellence and energy management for industry


Browse our articles, expert advices and clients testimonials: our experience save your time!
ETL-Extraire-Transformer-Charger-gestion-donnes

Wiki Blu.e: Extract – Transform – Load: why ETL software is essential to manage data

There are numerous types of data accessible at an industrial site. Whether originating from production equipment, maintenance alerts or external sources, these data are highly heterogeneous. Accordingly, how is it possible to aggregate, process and use them in order to optimise energy consumption? The ETL, a small software robot, can fulfil the job.

 

1. Definition

2. The scope of “Extract, Transform & Load”

3. The assets of a powerful ETL software

 

 

1. Definition

ETL is a small integration software program designed to extract (Extract phase) raw data from a source system, clean and prepare it (Transform phase) for subsequent loading (Load phase) into a database for later use. ETL software programs are widely used in industries due to the variety of data sources.

 

2. The scope of “Extract, Transform & Load”

# Extract – The “extractor” of the ETL software collects all data usable by the factory: volume and nature of manufactured products, system parameters on temperature, pressure and flow rates, machinery wear, energy consumption, along with external data such weather conditions, or raw material prices…

# Transform – The ETL “transformer” calculates parameters enabling correlations to be established between the plant’s heterogenous data, such as the energy consumption of a machine, quantities manufactured for each product, or the invoices charged by the energy supplier. The transform phase matches and correlates the data to deliver a general overview of the energy performance. The transformer also populates the base with useful data: thus if you want to trace a curve with points spaced every 10 minutes, even though the meter communicates only hourly data, the ETL software calculates the missing points via a linear or stepped regression. The transform computations can also “clean” the data using automatic learning rules, identify malfunctions or metering errors, called “outliers”, and replace them with relevant data.

# Load – Once cleaned, the data are then loaded into a database, also called a data warehouse, and organised via a programming interface. The data can then be processed by analytical SaaS software programs like blu.e pilot®. The data are hosted on several servers.

 

3. The assets of a powerful ETL software

The data that Blu.e customers analyse and use are thus generated by an ETL loading system. The ETL is a powerful and upgradable software designed specifically for industrial needs. It can handle millions of data lines and several dozen thousands of variables – which delivers gains in consistency and optimises the energy efficiency of their factories.