Overview

So many organisations are designing a new architecture for data processing. The introduction of new technology, the change of data usage, and the new regulations for data privacy have convinced organisations they need a new data architecture. Examples of new forms of data usage are data science, real-time data analytics, embedded BI, and customer-driven BI. Examples of new technologies are Hadoop, NoSQL, analytical SQ, Spark, and Kafka.

Sometimes a new data architecture is needed to fulfil the digital transformation dream or to become a more data driven organisation. Both terms imply that the organisation wants to exploit their data investment more intensely.

A new data architecture may also be required because the old data warehouse architecture cannot be extended anymore. It has reached its expiration data. And implementing a data lake isn’t always the right solution.

Therefore, numerous organisations need to design a new data architecture. But how? Where do you start? This tutorial explains all the aspects involved in designing a modern data architecture. What should be included in such an architecture? Is one high-level PowerPoint slide showing all the databases and data streams sufficient? What constitutes a good data architecture? Guidelines are given on the topics that should be included, including data streams and data stores, data quality, data security and privacy, governance, and metadata specifications.

The tutorial is based on years of experiences with designing modern and evaluating existing data architectures for all kinds of organisations, from small to large, and from non-commercial to commercial. Good and bad examples from real life situations are discussed as examples.

Topics:

  • Introduction – what is a Data Architecture?
  • Overview of New Technologies for Data Storage, Data Processing, and Data Analytics
  • Design Aspects for Data Architectures
  • Innovative New Data Architectures
  • Action Plan for Developing a Complete and Correct Data Architecture

All public courses are available as in-house training. Contact us for more information.

Learning Objectives

  • What are the steps to take to come up with the perfect data architecture? From requirement analysis via proof of concepts to a data architecture.
  • What is the importance of a holistic approach to analyzing technology, organization, and architecture in conjunction?
  • What are real life examples of new data architectures?
  • How can the new technology use optimally within a new data architecture?
  • How do you develop a data architecture?
  • Which components make up a data architecture?
  • What are the use cases, pros and cons of new technologies and how do they influence data architectures?
  • What is the value of well-known reference architectures, such as the Lambda architecture, the logical data warehouse architecture and the data lake?
  • What are the right criteria for a data architecture?

Course Outline

Part 1: Introduction – what is a Data Architecture?

  • Why a new data architecture?
  • Examples of real life data architectures
  • What are the key elements of a data architecture?
  • What are the differences between a data architecture and a solutions-architecture?
  • From batch via Lambda to the Kappa architecture
  • Benefits, drawbacks, and shortcomings of well-known reference architectures, such as the classic data warehouse architecture, the data lake, and transactional systems
  • From vision to implementation plan

Part 2: Overview of New Technologies for Data Storage, Data Processing, and Data Analytics

  • Benefits, drawbacks, features, and use cases of each technology
  • Data storage: analytical SQL, NoSQL, Hadoop, cubes
  • Data integration: ETL, data virtualization, data replication, data warehouse automation, enterprise service bus, API gateway
  • Data cleansing: home-made, professional
  • Data streaming: messaging, Kafka, streaming SQL
  • Data documentation: data glossary, data catalog, metadata management
  • Reporting tools: self-service BI, dashboards, embedded BI
  • Data science tools: programming languages, such as R and Python, machine learning automation tools, data science workbenches
  • Data security: anonymization, authorization

Part 3: Design Aspects for Data Architectures

  • First the technology or first the data architecture?
  • The importance of reusable transformation specifications for e.g. integration, filtering, correcting, and aggregation of data
  • Influence of specialized technology on data architectures
  • Why migration to the cloud: unburdening, high performance, scalability, available software?
  • Are all software products suitable for the cloud?
  • Design principles for dealing with data history and data cleansing
  • Modernization of a classic data warehouse architecture
  • Generating a data warehouse architecture with data warehouse automation tools
  • New requirements for transactional systems, such as storing historic data and continuous logging
  • The influence of GDPR: deleting customer data
  • Responsibility of data quality

Part 4: Innovative New Data Architectures

  • The logical data warehouse architecture as an agile alternative
  • Design rules, do’s and don’ts for a logical data warehouse architecture
  • From a single-purpose to a multi-purpose data lake
  • Requirements for implementing data science models, such as transparency, immutability, and version control
  • The changing role of the data lake: from data delivery system for data scientists to a platform for storing all the enterprise and external data
  • A data streaming architecture; when every microsecond counts
  • Technical challenges: performance, inconsistent data streams, storing massive amounts of messages for analytics afterwards
  • Operationalization of data science models
  • Merging data architectures to one unified data delivery platform
  • Differences between data hub and data warehouse
  • The data marketplace: from taylor-made to ready-made

Part 5: Action Plan for Developing a Complete and Correct Data Architecture

  • What is the business motivation for a new data architecture: ICT cost reduction, competitive improvement, new business model, new laws and regulations, improving reaction speed to business demands, or a more efficient exploitation of available data?
  • The importance of a business strategy and data strategy and the relationship with the data architecture
  • Who are the stakeholders and what is the C-level support?
  • Maturity level of the ICT organization
  • Description of the current data architecture; data flow, data storage, quantities, and technologies in use
  • Stock-taking of current bottlenecks; business and ICT, performance, functionality, costs, ICT organization and the immediate environment
  • Constraining rules, such as laws and regulations, budget restrictions, software limitations, and legacy systems.
  • Requirements and needs of the new data architecture; financial, available expertise, software, quantities, uptime, speed of data delivery, and level of unburdening.
  • Architecture and design principles
  • Current and future forms of data usage: standard reports, self-service BI, data science, customer-driven, mobile apps
  • Forms of data usage; batch, manual internally, manual extern ally, and sensors
  • Data types in use, including structured, unstructured, audio, video, text, and geo/gis.
  • Setting up the data architecture project; which choices must be made, which steps to take, is a PoC or Pilot required, what are key questions in a RfI, and convincing the organization

Part 6: Closing Remarks

Who It's For

  • Business Intelligence Specialists
  • Data Analysts
  • Data Warehouse Designers
  • Business Analysts
  • Data Scientists
  • Technology Planners
  • Technical Architects
  • Enterprise Architects
  • IT Consultants
  • IT Strategists
  • Systems Analysts
  • Database Developers
  • Database Administrators
  • Solutions Architects
  • Data Architects
  • IT Managers

Speaker

Rick van der Lans

Independent Analyst, Consultant, Author and Lecturer

R20/Consultancy

Rick van der Lans is a highly respected independent analyst, consultant, author, and internationally acclaimed lecturer specialising in data architectures, data warehousing, business intelligence, big data, and database technology. In 2018 he was selected the sixth most influential BI analyst worldwide by onalytica.com. He has presented countless seminars, webinars, and keynotes at industry-leading conferences. For many years, he served as the chairman of the annual European Enterprise Data and Business Intelligence Conference in London and the annual Data Warehousing and Business Intelligence Summit in The Netherlands. Rick helps clients worldwide to design their data warehouse, big data, and business intelligence architectures and solutions and assists them with selecting the right products. He has been influential in introducing the new logical data warehouse architecture worldwide, which helps organisations to develop more agile business intelligence systems. Over the years, Rick has written hundreds of articles and blogs for newspapers and websites and has authored many educational and popular white papers for a long list of vendors. He was the author of the first available book on SQL, Introduction to SQL, which has been translated into several languages with more than 100,000 copies sold. Recently published books are Data Virtualisation for Business Intelligence Systems and Data Virtualization: Selected Writings  He presents seminars, keynotes, and in-house sessions on data architectures, big data and analytics, data virtualization, the logical data warehouse, data warehousing and business intelligence.

IRM UK Public Courses via Live Streaming: 

Practical Guidelines for Designing Modern Data Architectures

New Big Data Storage Technologies: From Hadoop to Graph Databases, and from NoSQL to NewSQL

Fees

  • 1 day
  • £695
  • LIVE STREAMING - £695 + VAT (£139) = £834
Group Booking Discounts
Delegates
2-3 10% discount
4-5 20% discount
6+ 25% discount

Cancellation Policy:
Cancellations must be received in writing at least two weeks before the commencement of the seminar and will be subject to a 10% administration fee. It is regretted that cancellations received within two weeks of the seminar date will be liable for the full seminar fee. Substitutions can be made at any time.

Cancellation Liability:
In the unlikely event of cancellation of the seminar for any reason, IRM UK’s liability is limited to the return of the registration fee only. It may be necessary, for reasons beyond the control of IRM UK, to change the content, timings, speakers and date of the seminar.

Venue

This site uses cookies: Find out more.