This course explores the more advanced techniques for Data Modelling. In addition, techniques will be taught on how (and when) to create Data Models for non-relational solutions including Big Data together and the uses for data models beyond Relational DBMS development.
In the modern era, the volume of data we deal with has grown significantly. As the volume, variety, velocity and veracity of data keeps growing, the types of data generated by applications become richer than before. As a result, traditional relational databases are challenged to capture, store, search, share, analyse, and visualize data. Many companies attempt to manage big data challenges using a NoSQL (“Not only SQL”) database and may employ a distributed computing system such as Hadoop. NoSQL databases are typically key-value stores that are non-relational, distributed, horizontally scalable, and schema-free.#
Many organisations ask, “do we still need data modelling today?” Traditional data modelling focuses on resolving the complexity of relationships among schema-enabled data. However, these considerations do not apply to non-relational, schema-less databases. As a result, old ways of data modelling no longer apply.
This course will show Data modelling approaches that apply to not only Relational, but also to Big Data, NoSQL, XML, and other formats. In addition, the uses of data models beyond simply development of databases will be explored.
Prerequisite: Attendance at the Data Modelling Essentials class OR 3+ years of practical Data Modelling experience
Data Modelling Recap
Levels of Models
Data Modelling – Back to the Future?
Data Modelling didn’t start with relational! This may be a surprise to many people, but the first uses of data models were well before Relational data bases became the norm. The techniques are applicable to many of the modern non-relational formats we see today.
Modelling in the pre-relational days. We didn’t have RDBMS’s. We had Flat files, Sequential, VSAM, Hierarchical DBMS’s, Network DBMS’s, Inverted Architecture DBMS’s.
The techniques that were developed for these are directly appropriate to the NoSQL and Big Data world of today.
Data Modelling for Big Data & NoSQL
What has to change when we are developing data models for a Hadoop or other Big Data environment?
Do modelling tools support Big Data technologies, what are the restrictions and considerations?
What data modelling techniques are applicable when targeting a Big Data platform?
Does normalisation still have a place in the Big Data world?
Where’s our metadata in the model now?
In the age of big data, popular data modeling tools (eg ER/Studio, ERWin, PowerDesigner) continue to help us analyze and understand our data architectures by applying hybrid data modelling concepts. Instead of creating pure a relational data model, we now can embed NoSQL submodels within a relational data model. In general, data size and performance bottlenecks are the factors that help us decide which data goes to the NoSQL system.
NoSQL & Hadoop: How the 4 types of NoSQL databases still need data models, and how the ACID vs BASE paradigm affects this.
Modelling for Hierarchic Systems & XML
What must change when developing data models for XML & Hierarchic systems?
Services Oriented Architecture (SOA)
Why data models are essential for success.
Massively Denormalised Files:
Dimensional Data Models:
Application Packages & Data Models
Using Data Models for Data Integration & Lineage
How to exploit data models for design of data integration approaches and in data lineage.
Top Down Requirements Capture
When is it appropriate, what are the limitations.
Bottom Up Requirements Synthesis
How to Capture Requirements for Both Data and Process Needs
What comes first Data or Process – we’ll show the answer. The critical importance of understanding processes to get your data models right (and vice versa). Interaction between process and data models. Approaches for capturing Process AND Data Requirements.
Checking the Data vs the MetaData; Why Does It Matter?
Use of Standard Model Constructs and Pattern Models:
Different Data Modelling Notations & a Comparison Between Them
Normalisation: Progressing beyond 3NF. 4NF, 5NF Boyce-Codd, and why, and when to use them.
At the end of the course, delegates would have gained the following:
Build conceptual and logical data models, and know about compromises for physical design;
Level Set Understanding & Terminology:
Learn about the need for and application of Data Models in Big Data and NoSQL environments
Learn the best practices for developing Data models for Big Data and NoSQL environment
Practitioners who will need to read, consume or create data models, particularly for Big Data and non-RDBMS environments. Users who wish to gain a better understanding of data during Information Management initiatives including: