Learn Data Modeling

LearnDataModeling.com

Tutorial on Data Modeling, Data Warehouse & Business Intelligence!

  • Home
  • Training
  • Business Process
  • Data Modeling
    • Data Modeling Overview
    • Data Modeling Tools
    • Data Modeling Relationships
    • Data Modeling Types
    • Data Modeling Comparison
    • Physical Data Modeling
    • Modeling DW and Data Mart
    • Data Modeling Interview Questions
  • DM & Database
  • DW & ETL
    • DW & ETL Tutorial Topics
  • Software & Mobile Apps Testing
    • Software & Mobile Apps Testing Topics
  • BI
    • Business Intelligence Topics

Tag: metadata repository

Role of Metadata in ETL

October 15, 2015 learndmdwbi Leave a comment

Role of Metadata in ETL:

When you deal with a data warehouse, various phases like Business Process Modeling, Data Modeling, ETL, Reporting etc., are inter-related with each other and they do contain their own metadata. For example in ETL, it will be very difficult for one to extract, transform and load source data into a data warehouse, if there is no metadata available for the source like where and how to get the source data.

Let us explain the role of metadata in the ETL process with the help of an example table shown below which contains information about an organisation’s employees.

Employee NameEmployee AgeEmployee SalaryEmployee Title
John Hick36$3000Informatica Specialist

In the above table, the second row, containing information like John Hick, 36, $3000, Informatica Specialist are known as Data. Whereas the first row, (i.e) table header containing headings like Employee Name, Employee Age, Employee Salary, Employee Title are called as Metadata for the above said data.

An organization may be using data modeling tools, such as Erwin, Embarcadero, Oracle designer, Sybase Power Designer etc., for developing data models. Functional and technical team should have spent much time and effort in creating the data model’s data structures (tables, columns, data types, procedures, functions, triggers etc.,). By using metadata capturing tools, these data structures can be imported into metadata repository which we call it as metadata.

For example when you work with Informatica’s Metadata Exchange, it captures the metadata present in these tools and loads into the repository. There is no need for Informatica developer to create these data structures once again since metadata(data definitions) have been already captured and stored. Similarly most of the ETL tools have that capability to capture metadata from RDBMS, files, ERP, Applications etc.

In ETL, Metadata Repository is where all the metadata information about source, target, transformations, mapping, workflows, sessions etc., are stored.
From this repository, metadata can be manipulated, queried and retrieved with the help of wizards provided by metadata capturing tools. During the ETL process, when we are mapping source and target systems, we are actually mapping their metadata.

In any organization, a useful metadata often stored in a repository can be a handy resource to know about the organization’s information systems. Assume that each department in an organization may have a different business definitions, data types, attribute names for the same attribute or they may have a single business definition for many attributes. These anomalies can be overcome by properly maintaining metadata for these attributes in the centralized repository.

Thus metadata plays a vital role in explaining about how, why, where data can be found, retrieved, stored and used efficiently in an information management system.

 

  • BI

Data Modeling Training using Erwin

Course #1:
How to use Erwin for Data Modeling!

Course #2:
Providing Solutions/Assistance to Data Modeling Projects!

Course #3:
Online Advanced Data Modeling Training!

For Syllabus and other details, please click here.

Training@LearnDataModeling.com
(or) 91-9080157239

Categories

  • BI (11)
  • Business Process (16)
  • Data Modeling (53)
    • Data Modeling Comparison (3)
    • Data Modeling Interview Questions (3)
    • Data Modeling Overview (11)
    • Data Modeling Relationships (5)
    • Data Modeling Tools (11)
    • Data Modeling Types (9)
    • Modeling DW and Data Mart (9)
    • Physical Data Modeling (2)
  • DW & ETL (20)
  • Software & Mobile Apps Testing (15)
  • Training (2)
  • Uncategorized (1)

Recent Posts

  • Online Data Modeling Training – Crash Course
  • Online Data Modeling Training Syllabus
  • Business Intelligence Tutorial
  • Advantages of Multi Dimensional Databases
  • OLAP and OLAP Hybrids
  • Privacy Policy
  • Disclosure Policy
Powered by WordPress and Dynamic News.
error: Content is protected !!