Data Virtualization From A to Z

Rick van der Lans, R20/Consultancy

Data is increasingly becoming a crucial asset for organizations to survive in today’s fast moving business world. In addition, data becomes more valuable if enriched and/or fused with other data. Unfortunately, enterprise data is dispersed by most organizations over numerous systems all using different technologies. To bring all that data together is and has always been a major technological challenge. In addition, more and more data is available outside the traditional enterprise systems. It’s stored in spreadsheets, simple file systems, cloud applications, in weblogs, in social media systems, and so on.

This is where data virtualization comes to the rescue. Data virtualization is a technology that makes a heterogeneous set of databases and files look like one integrated database. When used in business intelligence systems, it can make the architectures dramatically simpler, cheaper, and, most importantly, more agile. New reporting and analytical needs can be implemented faster and existing systems can be changed more easily.  This half-day workshop explains in detail what data virtualization is, how the products work, advantages and disadvantages are discussed, products are compared, and focuses on aspects, such as query performance, caching, data security and data integration.

• Under the hood of a data virtualization server
• Importing non-relational data, such as XML and JSON documents, web services, NoSQL, and Hadoop data
• Query optimization techniques
• Caching for performance and scalability
• Securing access to data in virtual tables
• Design guidelines and tips and tricks
• Market overview, including AtScale, DataVirtuality, Denodo Platform, Dremio, FraXses, IBM Data Virtualization Manager for z/OS, Red Hat JBoss Data Virtualization (Teiid), Stone Bond Enterprise Enabler Virtuoso, Tibco Data Virtualization

Principles for the New BI

Donald Farmer, Principal, TreeHive Strategy

Whether we work in business or IT, it sometimes feels like we are bombarded with advice about which tools or platforms to choose for data analytics. But business insights don’t arise from features and functions and choosing a good platform is only the start of your analytics journey.

Too often we overlook the need for some basic principles – ways of thinking and evaluating data, technologies and organizational needs that act as landmarks on our path to a culture of analytics. For example, it’s critical to remember that the heart of business analytics is still decision-support – losing sight of that principle can waste a lot of time and money! What about the relationship between data preparation and data analysis? We need to keep in mind how closely entwined these techniques are, if we are to be effective in either.

In addition to well-established practices, as new technologies emerge we need new, relevant, principles to guide us through machine learning and AI. In this course, Donald Farmer will set out 10 fundamental principles of modern data analytics—perceptive, provocative ideas about how data really works in our businesses. These principles provide valuable starting points for planning, evaluating, and promoting business intelligence projects.

You Will Learn …

• Why understanding the human element is essential to good information design
• How machine learning and AI are already changing the nature of business knowledge
• Who makes decisions in a world of automation
• Why good governance does not necessarily result in good decisions
• The significance of bias in machine learning, but also in everyday analytics.

This workshop is suitable to:

• BI and analytics architects designing and developing analytic systems
• Business leaders trying to guide teams through the important changes happening in analytics and machine learning
• Data analysts and data scientists who need to work and communicate with business users
• IT leaders investing in platforms, tools and training in analytics and machine learning.

Artificial Intelligence for Mere Mortals

Jos van Dongen, Principal Consultant, Tholis Consulting

The evolution in attention from Big Data to Machine Learning to Artificial Intelligence completes the transition from mostly technology focused to human centric computing. The net effect however is that while the applications of the technology are well known to many, the technology itself is becoming less transparent and understood by only a few. As a result, many people shy away from adopting new algorithms and use cases simply because it looks too complex or even scary. This workshop will break down the latest advances in AI like chatbots, decisioning engines and image recognition into the different components that make up the technology. Using a use case driven approach, you’ll get deeper insights and a good understanding of how AI actually works. You will learn:

• How organizations in healthcare, insurance, retail and transportation are using AI
• What the power and limitations of AI are
• How to select the AI building blocks you can use in your own projects

Getting Started with Data Quality – A Primer

Jon Evans, Equillian

Today, more than ever, the quality of data, underpinned by a robust approach to Data Quality Management, is critical to the success of every organisation. Unfortunately, it is a topic that is still impenetrable to many through the use of unfamiliar jargon and too much emphasis on technology.

In this half-day workshop, Equillian’s Jon Evans will seek to redress the balance, by taking the audience on a journey from first principles right through to advice on establishing a Data Quality Programme. Along the way, both beginners and those already familiar with the topic will benefit from a business-focused approach, based on industry best practice coupled with many years of experience helping organisations tackle their Data Quality challenges.

The session will be structured around 4 key topics:

• Why should I care about data quality?
• Monitoring data quality
• Improving data quality
• Developing a DQ Programme

Beyond GDPR Compliance to Ethical Data: A Practical Introduction to Ethical Information Management

Katherine O’Keefe, Lead Consultant / Chief Ethics Oficer, Castlebridge

Your organization is already doing “ethics” whether you explicitly consider it or not.  The question is whether it is a “best efforts” production without the clear guidance of principles. Increasingly data ethics is becoming recognized as important not just as a “fuzzy” concept of corporate social responsibility but as both a commercial differentiator and as a real influence on our day to day experience and a key management risk.  Have you taken the time to consider what it is you’re doing? Is your organization’s leadership setting the tone from the top?  Do you have appropriate systems of management in place to support ethical decisions and actions? This course is designed for information management professionals and provides a detailed framework and practical tools and techniques for implementing an ethical information management strategy.

Key takeaways for this session include:

• An overview of fundamental Ethical Concepts as related to Information Management
• Risk management, Information management practices
• Using GDPR Compliance to focus Ethical Data use beyond compliance
• Methods to align ethics with Information Governance
• Practical tools for implementing an ethical information strategy

Data Modelling for Big Data

Chris Bradley, Data Management Advisors

In the modern era, the volume of data we deal with has grown significantly.  As the volume, variety, velocity and veracity of data keeps growing, the types of data generated by applications become richer than before. As a result, traditional relational databases are challenged to capture, store, search, share, analyse, and visualize data.  Many companies attempt to manage big data challenges using a NoSQL (“Not only SQL”) database and may employ a distributed computing system such as Hadoop.  NoSQL databases are typically key-value stores that are non-relational, distributed, horizontally scalable, and schema-free.

Many organisations ask, “do we still need data modelling today?”  Traditional data modelling focuses on resolving the complexity of relationships among schema-enabled data.  However, these considerations do not apply to non-relational, schema-less databases. As a result, old ways of data modelling no longer apply.

This workshop will show Data modelling approaches that apply to not only Relational, but also to Big Data, NoSQL, XML, and other formats. In addition, the uses of data models beyond simply development of databases will be explored.

Data Modelling – Back to the Future? 

• Data Modelling didn’t start with relational! This may be a surprise to many people, but the first uses of data models were well before Relational data bases became the norm. The techniques are applicable to many of the modern non-relational formats we see today.
• Modelling in the pre-relational days. We didn’t have RDBMS’s.  We had Flat files, Sequential, VSAM, Hierarchical DBMS’s, Network DBMS’s, Inverted Architecture DBMS’s.
• The techniques that were developed for these are directly appropriate to the NoSQL and Big Data world of today.

Data Modelling for Big Data & NoSQL

• What has to change when we are developing data models for a Hadoop or other Big Data environment?
• Do modelling tools support Big Data technologies, what are the restrictions and considerations?
• What data modelling techniques are applicable when targeting a Big Data platform?
• Does normalisation still have a place in the Big Data world?
• Where’s our metadata in the model now?

Designing A Logical Data Warehouse

Rick van der Lans, R20/Consultancy 

Business intelligence has changed dramatically the last years. The time-to-market for new reports and analysis has to be shortened, new data sources have to be made available to business users more quickly, self-service BI and data science must be supported, more and more users want to work with zero-latency data, adoption of new technologies, such as Hadoop, Spark, and NoSQL, must be easy, and analysis of streaming data and big data is required.

The classic data warehouse architecture has served many organizations well. But it’s not the right architecture for this new world of BI. It’s time for organizations to migrate gradually to a more flexible architecture: the logical data warehouse architecture. This architecture, introduced by Gartner, is based on a decoupling of reporting and analyses on the one hand, and data sources on the other hand. With the logical data warehouse architecture new data sources can hooked up to the data warehouse more quickly, self-service BI can be supported correctly, operational BI is easy to implement, the adoption of new technology is much easier, and processing of big data is not a technological revolution, but an evolution. And most importantly, the technology to create logical data warehouses is available: data virtualization servers. In this practical tutorial, the architecture is explained in detail. Tips and design guidelines are given to help make this migration as efficient as possible.

• The benefits of the logical data warehouse architecture and differences with the classic data warehouse architecture
• How easily new data sources can be made available for analytics and data science
• How self-service analytics can be supported by a logical data warehouse, and how it helps to share specifications across different analytics tools
• How your organization can successfully migrate to a flexible logical data warehouse architecture in a step-by-step fashion
• How logical data warehouses help integrate self-service analytics with classic forms of business intelligence
• The real-life experiences of organizations that have implemented a logical data warehouse

The Art of AI

Jan W Veldsink, Lead AI at Rabobank Compliance and Core Teacher, Nyenrode / Rabobank

You will not have failed to notice the rise of Artificial Intelligence after a long winter into a sunny spring. It is now time to find out what the areas of BI and AI have to offer each other. In this session, because AI is being used more and more recently, Jan Veldsink will pay attention to what AI has to deliver organizations in a BI context.

AI currently focuses on Deep Learning, a layered variant of Machine Learning based on Neural Networks. By training a network with examples it forms a competence, usually classifying, for example, images or events. Requirements are set for the training examples and we will see how BI can play a role in making suitable training materials.

Or you can be assisted as a Data Scientist by an already trained AI! We will see several examples of AI-based systems that can take over part of the work of Data Scientists.

And for the real connoisseur there is the possibility to train an AI with own examples and then have them do part of the BI work. Typical examples of this are anomaly detection in fraud investigation and determining when machinery maintenance should take place. We will see various platforms that make it possible.

Governing the Data Lake - The Critical Importance of An Information Catalogue

Mike Ferguson, Intelligent Business Strategies

With so much new data being captured across the enterprise and multiple self-service and data science initiatives being undertaken, something has to know and track what’s going on and what’s available in an increasingly complex data landscape.  In addition, people need the ability to publish what data and what artefacts (ETL jobs, data preparation jobs, analytical models, dashboards, etc) currently exist to encourage re-use and prevent re-invention.  This session shows how information catalogue software can be used to publish data, artefacts and policies to manage, organise and govern in a multi-platform data and analytical environment.  This session will cover:

• What is an information catalogue?
• Information catalogue capabilities, e.g. business glossary, automatic data profiling, automatic tagging and data classification, automatic sensitive data discovery, automatic data indexing, faceted search, data marketplaces, artefact publishing
• How does an information catalogue help govern a data lake?
  • Data meaning
  • Data Quality
  • Data access security
  • Data Privacy
  • Data Lifecycle management
  • Data Lineage
• Information Catalog technology offerings
• Creating a governed information value chain using an information catalogue
• Key roles and responsibilities – Information producers, information consumers and governance
• Publishing data and analytics as a service
• Integrating disparate metadata via Open Metadata and Governance
• Policy management and policy enforcement across multi-platform via an information catalog
• Integrating the catalog with data management, data science, and BI technologies
• Consumer trust – Accessing business glossaries and metadata lineage

Making Enterprise Data Quality a Reality

Nigel Turner, Principal Information Management Consultant EMEA, Global Data Strategy 

Many organisations are recognising that tackling data quality (DQ) problems requires more than a series of tactical, one off improvement projects. By their nature many DQ problems extend across and often beyond an organisation.  So the only way to address them is through an enterprise wide programme of data governance and DQ improvement activities embracing people, process and technology. This requires very different skills and approaches from those needed on many traditional DQ projects.

If you attend this workshop you will leave more ready and able to make the case for and deliver enterprise wide data governance & DQ across your organisation. This highly interactive workshop will also give you the opportunity to tackle the problems of a fictional (but highly realistic) company who are experiencing end to end data quality & data governance challenges. This will enable you to practise some of the key techniques in a safe, fun environment before trying them out for real in your own organisations.

Run by Nigel Turner of Global Data Strategy, the workshop will draw on his extensive personal knowledge of initiating & implementing successful enterprise DQ and data governance in major organisations, including British Telecommunications and several other major companies.  The approaches outlined in this session really do work.

The workshop will cover:

• What differentiates enterprise DQ from traditional project based DQ approaches
• How to take the first steps in enterprise DQ
• Applying a practical Data Governance Framework
• Making the case for investment in DQ and data governance
• How to deliver the benefits – people, process & technology
• Real life case studies – key do’s and don’ts
• Practice case study – getting enterprise DQ off the ground in a hotel chain
• Key lessons learned and maxims for success

Closing the Communication Chasm: Using Stories to Convey Actionable Insights

Lori Silverman, Partners for Progress

What’s the best way to communicate data to accelerate decision-making and action? Here’s a proven path: Reveal insights hidden in the data. Storify the insights that need to be given voice. And determine the best way to communicate those stories.  What does this imply? You need to know how to shift conversations from questioning data, data integrity, data analyses, and data visualization to dialogue grounded in foresight, strategic thinking, and action. And you need to know how to craft compelling meaning-filled stories that don’t contain data. What???? No data??? This workshop demonstrates how to unlock value for you and your organization through becoming a data translator and teaching others how to do the same.

• Determine which questions to ask to move people from data to insight to action.
• Identify three types of insights and stories that may reside within a set of data.
• Outline compelling stories that move people to action.

The Lost Art of Conceptual Data Modelling

Chris Bradley, Data Management Advisors

The lost art of Conceptual Data Modelling …. the essential position of the conceptual data model & interdependency with a conceptual process model.

Data Models are NOT just for Database design.  The vital role of the conceptual data model is misunderstood and frequently overlooked.  The CDM is vital in understanding the data concepts of the organisation to get a great understanding of what the data stuff is – irrespective of the IT systems.  The CDM also provides a vital pivot for Data Governance.  And furthermore, since end-to-end business processes use data, the CDM has a critical role in anchoring the processes model and ensuring alignment and consistency.

• What is a Conceptual data model? Guidelines for level and content of a Conceptual Data Model.
• How to capture robust requirements for both Data and Process needs.
• Uses of a Conceptual Data Model: Data Governance, Business communication and Process Alignment
• Improving communication between modellers and business stakeholders, & how to use high-level data models to aid communication (and when not to).
• Presenting data models to business users and how to conduct feedback sessions. A data model quality checklist
• Data modelling is NOT just for RDBMS’s:  Case studies on other uses.

Plenary Keynote: It's Not about You, It's About Them: Helping Others Take Action Based on Data

Lori Silverman, Partners for Progress

Maybe you routinely collect, cleanse, mine, and monitor data for insights. Or select technologies for data storage, processing, and transportation. Perhaps you get requests for data and analyses that fuel all sorts of visualizations. Or you’re implementing big data governance structures. These all have one goal in common: Enabling quicker and better organizational decision-making that sparks collaborative and aligned action. This begs two questions: Why does a communication chasm exist between those who do this work and those who need to solve pressing business problems and implement the solutions? And how do emotion and intuition reside harmoniously with data? This keynote presents a model and how-to framework for closing these gaps.

Objectives:

• Illustrate the relationship between data, insight, intuition, and action.
• Identify three different kinds of insight that can emerge from data.
• Demonstrate why story is the narrative vehicle for decision making and action.

Business Intelligence & Analytics Keynote: Working with Ambiguity

Donald Farmer, Principal, TreeHive Strategy

We have all seen sales forecasts where next year’s number are projected down to the last penny. Many of our efforts in Business Intelligence suffer in a similar way from false precision. In today’s workplace, these old certainties are wearing away. With the growth of Artificial Intelligence, Machine Learning and Predictive Analytics, we need to become much more comfortable with the language of probability and inference.

In this Keynote, Donald Farmer will explore hw business users can become more comfortable with ambiguous information. We’ll see how new styles of visualization and reporting can help to make predictive analytics more actionable. And we’ll look at decision making styles and organizational patterns that are more appropriate for this more complex world.

Enterprise Data Keynote: Using Enterprise Information Management at the International Criminal Court to End Impunity

Dr Jones Lukose, Information Management Officer, The International Criminal Court

Human existence has continuously sought to find information, use and preserve it for posterity. Information preservation is, therefore, one of the most important tasks of communities and organizations. Information is now more correlated than ever and found in large quantities known as ‘big data’; it is pervasive, difficult to capture, store or analyze.

In this Keynote, the International Criminal Court (ICC), a judicial organization that has the preservation of its information as a critical aspect of its judicial obligation, is discussed. Born in the digital age, the Court has adopted an eCourt strategy covering all aspects of its operations to end impunity in the world. And to solve the accountability challenge its functions are redefined towards digital information orchestration and less towards traditional paper preservation.  In this Keynote, therefore, a story is told in the form of three buttons and in it address ICC’s information management practices and tools.

Key Topics:

• Information value and accountability
• Records managers and archivists as part of the information system
• Probabilistic and unpredictable
• Intuition and Trust
• Records managers and archivists as orchestrators
• Big Data

Take Away Include:

• Understand the Court and its global mandate to end impunity
• Learn what it means to run an eCourt
• Understand the new role of records and archives professionals in a modern institution
• Understand the motivational dimension of managing enterprise information in a “digital” organisation

The act of preservation of elements of human existence does not only feed to aesthetic functions but also to the fulfilment of accountability needs of the society. In this digital culture of ‘data everywhere,’ the demand for accountability is high necessitating a redefinition of information management practices. This is, therefore, a story of how information management practitioners in the ICC are redefining their relevance and value while helping the organisation end impunity.

Plenary Keynote Panel Discussion: The Changing Role of Data in Organisations

Moderator: Rick van der Lans, R20/Consultancy

We all know that the role of data within organizations has changed? There was a time when enterprise data was deeply hidden in our data centers. It was very hard for business users to get access to it. Nowadays, data plays a key role in day-to-day operations. Without the right data at the right time and in the right form, businesses operations may grind to a halt. New software and hardware technologies have made it possible to exploit data in many more ways than before. This all sounds easy, but what are the consequences of this changing role? What does it mean for ourselves, the BI specialists? And will an extensive use of data clash with the GDPR? Are we still using the right technology? Do we have the right data architecture? We are still struggling with many of these and similar questions. All these questions and more will be debated during this panel.

The Automated Factory Worker

Santanu Dawn, Master Data Leader EMEA, Goodyear 

The terms Dig Data, Machine Learning and IOT are the buzzwords of the moment. There is much discussion on what they are and why it is different from other projects. Now welcome to the world of manufacturing. For ages, our machines have been generating huge amount of data, it’s only now that we have the capability and technology to cost effectively capture and process it. In pure statistical terms, the manufacturing world generates as much machine data as the social media. The question is what to do with it.

In this session, we will try to build an optimized manufacturing organization by:

• Beating the bottleneck – The new era of analytics
• Augmenting workforce through technology
• Intelligent software systems and how they are transforming knowledge work
• New advances in artificial intelligence, machine learning and natural user interfaces

Data Product Development: from Zero to Hero

Andrey Sharapov, Data Scientist, Lidl

Modern organizations are overwhelmingly becoming convinced that data can be successfully turned into better decision making that will directly result into higher profits and lower costs. At Lidl, we believe that data products are the key ingredient that can either automate or supplement business processes. Ability to give internal decision makers an opportunity to explore not only the landscape of their business with descriptive analytics but also to allow them to uncover hidden patterns and dig deeper using prescriptive analytics is becoming increasingly popular among different organizations.

Needless to say that starting from scratch is a very difficult endeavour  and often companies begin with simple ad-hoc analysis but fail to progress toward data products due to lack of talent and the ever-increasing technological complexities. Indeed, a data product is a piece of software that is able to consistently produce meaningful results, while ad-hoc analysis is usually a single and irreproducible dive into data with doubtful outcome.

Lidl is a German global discount supermarket chain, that operates over 10,000 stores across Europe and the United States. In this session, Andrey will show you how to build a data product starting literally from pieces of ad-hoc research.  He will demonstrate their experience in working with internal customers and the challenges that data science teams face. Since they are a young team of data engineers and data scientists, Andrey will share how they jump-started data product development at Lidl and show the key achievements as well as challenges that they came up against. In this talk, Andrey will illustrate how they tamed their Hadoop cluster,  took advantage of the Hortonworks platform,  wrote and deployed production code in R.

Data, Meet Regulations. How do you do?

Peter Campbell, Founding Member & Director, DAMA BeLux

In recent years, regulations have increased significantly, both here in Europe as well as globally. Some of these were triggered by the global financial crisis of 2008, whilst others are driven by objectives of market transparency, safety, or considerations for privacy of personal data.

In this session, several regulatory business cases will be presented, and, wherever possible, related to the need for improvements in data management, not only to support compliance, but to improve overall data management capabilities and effectiveness, in specific industries and across organisations.

Representative compliance business cases include:

Energy/High Power Electricity Transmission:

European Commission Regulation 543/2013 (of 14 June 2013) on the submission and publication of data in electricity markets);

Pharma / Life Sciences:   IDMP (IDentification of Medicinal Products):

In Europe: European Commission Commission Implementing Regulation (EU) No 520/201  (articles 25 and 26) which obliges European Union (EU) Member States, marketing authorisation holders and EMA (European Medicines Agency) to implement the standards developed by ISO (11238, …..)

In the U.S:  Compliance / regulation is applicable and enforceable by the FDA (Food and Drug Administration)

GDPR (General Data Protection Regulation, EU 2016/679 [14 April 2016, effective 25 May 2018]):

Regulation on the protection of natural persons with the regard to processing of personal data and on the free movement of such data;  It also addresses the export of personal data outside the EU and EEA (European Economic Area).  The GDPR aims primarily to give control to citizens and residents over their personal data and to simplify the regulatory environment for international business by unifying the regulation within the EU.

There will also be discussions about some of the international standards organisations and specific standards, which often play important roles in these compliance projects; for example, this will include standards of the IEC (International Electrotechnical Commission), the ISO (International Standards Organisation), and also the W3C (World Wide Web Consortium).

Wherever applicable, the DMBOK2 (Data Management Body Of Knowledge) will be discussed, and new data management services and improvements of this version of the DMBOK (released in 2017) will be highlighted, for the benefits this framework can bring, to facilitate regulatory / compliance projects, as well as well as to improve overall data management environments.

Why Training the Organisation and not just the Data Team is Vital

Phil Yeoman, Head of Data Governance, The Pensions Regulator

In this ever changing environment of data it’s easy to focus on the last bit of tech kit you have persuaded procurement to purchase and/or the skills of the data scientist that you have just recruited – Do you go Data Lake or Data Warehouse.  Let’s fire up an Hadoop Environment and  sandbox away.

But have you stepped back and thought about that gap between what we data folk talk about and the business folk understand?

Phil’s talk will focus on:

• Bridging the gap between data people and business people.
• Why training the organisation and not just the data team is vital.
• Why people and culture matter

Delegates will learn:

• How language and concepts can be intimidating
• Why people need to have faith and confidence in what data teams do
• If organisations want to be data driven all their people need to be data savvy.

How CDOs & CIOs are Driving Digital Transformation

Ajay Khanna, Vice President, Marketing

Today’s business landscape more dynamic than ever. New revenue models, stringent regulations, and high customer expectations are forcing organisations to evolve or face being overrun by more nimble competitors.

CDOs and CIOs of established business are looking to digital transformation as a key initiative. But what exactly does digital transformation entail? At its core, any digital transformation requires clean and consistent data, reconciled across systems and channels. An enterprise-wide data management foundation that ensures real-time access to reliable data of all types at scale and is non-negotiable. Data access must be democratised across all groups and divisions so that teams can get a 360-degree view of customers, products, organisations and more. However, it’s not just about disconnected siloed analytics. It’s about the next generation of operational data-driven applications that allow frontline business users to gain relevant insight and intelligent recommended actions so they can achieve their goals. This session explores how some of the largest companies in the world are transforming themselves using the same modern data management technology used by Internet giants such as Amazon, Facebook, LinkedIn, and Google.

Blockchain as Data Processing Railroads

Hossein Kakavand, CEO and co-founder, Luther Systems

Enterprise processes , within and between organizations, are fragmented due to focus on functions over processes, leading to duplication of effort, propagation of error, need for reconciliation of data across processes, which is costly and limits the streamlining of these processes and new potential applications.

We talk about the sources of fragmentation, how Smart Contracts can address these limitations by providing railroads for enterprise process which sit along side enterprise systems not replace them, the benefits of smart contracts, the interplay between smart contracts and artificial intelligence.

(i) Smart Contracts address enterprise process fragmentation by providing railroads for enterprise process

(ii) these railroads sit along side enterprise systems not replace them,

(iii) the benefits of smart contracts

(iv) the interplay between smart contracts and artificial intelligence

Plenary Keynote: Data Science for Grown Ups: How to Get Machine Learning out of the Lab to Scale it Across the Enterprise

Dr Alex Borek, Global Head of Data & Analytics, Volkswagen Financial Services AG

Digital players like Amazon, Uber and Netflix are using data science and machine learning at a large scale to drive business value in all of their core business processes. Many large organisations in more traditional industries have also invested heavily in data science, big data, data governance and business intelligence over the past few years and often struggle to scale their successful machine learning projects beyond a small pilot scope. In other words, algorithms stay in the lab and are not put into the heart of the enterprise. This keynote presentation highlights experiences and strategies in maturing a data lab into a global data factory organisation that can ensure that the promised value of data science and machine learning is truly realised. It also discusses the organisational implications for business intelligence, data management and data architecture and the role of cloud driven technologies in the required transformation to get your company ready for the age of artificial intelligence.

Key take aways:

• Learn about the organisational implications for scaling data science and machine learning to the heart of the enterprise
• Understand the role of business intelligence, data management and data architecture in the business transformation
• Hear about good practices and strategies for managing the organisational and technological change process towards an AI driven enterprise

Business Intelligence & Analytics Keynote: Making Money with Customer Data

Sakari Jorma, Head of Business Technologies, Chief Digital Officer & CTO, Talenom

When a company’s EA and data management reaches to the level of maturity where it becomes an asset, how many companies monetize this? Customer activity data and consumer profiling are here to stay and yes, many do this, but how about truly creating value to existing customers and making more money to your company? How does one ramp up a new Digital business using BI? What is the roadmap? What has to be in plan before one can jump to the world of SAAS business with customer data?

• EA in good shape
• Next to perfect BI tools
• SAAS business model
• USE CASES – value creation to customer – why should I buy this?
• Transition from conventional sales to Up & Cross sales models

Talenom is the one of the largest accounting and financial services company in Nordics, which reported aggressive over 18% turnover increase Q1 2018 and continues to expand. The company is leading in the digital accounting and automation in the region, which has led to operating margin of 18%. Their story begins from “basics in place in IT” and understanding the data as an asset. From this, they went to customer value process and quick implementation of SAAS strategy.  With customer value cases combined to data and tools, the creation of BI business was obvious.

Talenom set up a Business Technologies business unit, which combines the IT, Automation, Integration, customer support and Up and cross sales functions, led by CDO.  Digital business is not IT or Head of BI job – it’s a job for the CDO!  This is a true opportunity for the future BI and Chief Data officers.

Enterprise Data Keynote Panel - GDPR: Beyond Compliance

Moderator: Mike Simons

We are living in the new world of GDPR, with most organisations proudly announcing that from 25 May they have been compliant with the new privacy regime.

Behind these announcements, however, is a deep concern about whether the effort that went in to achieving compliance is sustainable and how to demonstrate and deliver long-term business benefits from your compliance efforts.

GDPR is not a one-off test that an organisation passes or fails, it has to be embedded into the enterprise and its day-to-day practices.

Join this panel discussion and:

• Explore ways of getting real business value from compliance
• Examine how to get use compliance programmes to get actionable business intelligence from the data you hold
• Using GDPR and transparency over data as a business differentiator
• The scope for automation in GDPR-related data management programmes
• How organisations are maximising the value of data lineage

Enterprise and Self-Service BI on Top of a Data Lake

Krystyna Kurinna, Teamlead Data Access Services & Solutions, Scout24 AG

The Data Lake concept has been constantly developed during past years. How to make sure all business users can access data easily and carry on performable and sustainable analysis? I would like to share our experience about building enterprise and self-service BI on top of a data lake and cover different aspects of building data-driven culture:

• Structuring your teams as data platform teams (to make sure data lake works and data is accessible) plus two analytics teams (central and decentral) is a successful set up for working BI on Data Lake
• To make sure a scalable and agile business intelligence is in place, data producers take full responsibility for data publishing and quality, and data consumers – for metrics definition and implementation of business logic layer
• Additional technical skills like ability to build a data pipeline are required for powerful analysis and are indispensable for new generation of data analysts

Data Science Workbenches and Machine Learning Automation – New Technologies for Agile Data Science

Mike Ferguson, Managing Director, Intelligent Business Strategies

The demand for analytics is now almost everywhere in the business. Analytics are needed in sales, marketing and self-service, finance, risk, operations, supply chain and even HR. However, the current shortage of data scientists and the reliance on detailed skills such as programming, has led many corporate executives to question current approaches to development of high value analytical models and ask if they can be accelerated in any way to improve agility and reduce time to value. This session looks at this problem in detail and at how emerging data science workbenches and machine learning automation tools can help reduce the reliance on highly skilled data scientists and allow business analysts to become data scientists and so meet the demand of business

• The explosion in demand for analytics
• Data science and the modern analytical ecosystem
• Challenges with current approaches to analytics
• Requirements to reduce time to value and accelerate development of analytical models
• Improving productivity by integrating Information catalogs and data science workbenches e.g. Cloudera Data Science Workbench, IBM Watson Studio
• Accelerating model development, monitoring and model refresh using ML automation tools, e.g. DataRobot, Tellmeplus Predictive Objects, SAS Factory Miner, Dataiku Data Science Studio
• Facilitating rapid analytics deployment via analytics as a service to maximise effectiveness and competitive edge

You Cannot Inspect Ethics into a Product: Ethics and Quality Management

Katherine O’Keefe, Lead Consultant / Chief Ethics Officer, Castlebridge

“It’s not enough to do your best; you must know what to do and then do your best”.

Ethics are big news in the headlines recently.  Whether we’re looking at remarkable failures in ethics the ethical implications in Big Data processing or self-driving cars, it’s becoming more and more clear that we need to get this right. The challenge arises in moving from abstract discussion to practical application.

This session uses Deming’s view of quality as a starting point to understand and implement ethics in Information Management. You’re already doing “ethics” whether you explicitly consider it or not.  The question is whether it is a “best efforts” production without the clear guidance of principles. Have you taken the time to consider what it is you’re doing? Is your organization’s leadership setting the tone from the top?  Do you have appropriate systems of management in place to support ethical decisions and actions?

Key takeaways for this session include:

• An overview of Ethics and their relevance to Information Management practices
• Three types of Normative Ethics in organizations
• How W. Edwards Demings’ 14 points can be a starting point for introducing an ethical framework
• An overview of practical methods to align ethics with Information Governance
• Risk management, Information management practices

Mind your Language: The Criticality of Common Data Definitions in Managing Complex Data

Becky Russell, National Lead for Data Standards, UK Environment Agency
Nigel Turner, Principal Consultant EMEA, Global Data Strategy

The Environment Agency generates, collects and processes large quantities of complex data to support its mission to improve the environment of England.  To enhance its management of this data it has embarked on a major programme to develop and enforce clear data definitions and data standards, driven in part by the UK Government’s Open Data agenda which makes much of the Environment Agency’s data available for public use and scrutiny.

This case study will highlight the importance of shared, common data definitions as a vehicle to improve collaboration in data management, both across the Agency and with its extensive network of partners.   It will cover:

• The role of the Environment Agency and the importance of data management
• The issues resulting from a lack of data definitions and standards
• How the Environment Agency is tackling the problems
• Why a common language is critical in collaborative data management
• The role of data science & data modelling in developing common definitions and standards
• The necessity of multiple definitions in complex data sets
• The relationship between data definitions and IT development
• Lessons learnt and advice for other organisations trying to implement data definitions and data standards

The Power of Data as a Catalyst for Collaboration

Jonathan Sunderland, Data Evangelist, Harbr

Today the challenge is not how much data you have, it’s how much data you can use effectively – making better decisions quicker.

A solitary data source is rarely sufficient for analytics or decision-making. Yet within organisations the combination of legacy & non-integrated systems, cloud offerings and spreadsheets become bewildering.  Add the complexities of external data and the challenges become truly daunting.

Where analysts can collaborate in an ecosystem where data is acquired and profiled once they can debate and iterate analysis as quickly as possible and establish a single version of truth for others to trust in decision making.

Serendipitous discoveries are far more likely, conversations spark curiosity and it is easier for analysis to truly drive innovation – data can become a true catalyst for collaboration.

What delegates will learn:

• Why customers expect organisational agility
• The importance of collaboration at the raw data level
• How data can be the catalyst for change
• What does a data cultured organisation feel like

Brewing a Data Driven Organisation Leveraging on Self-Service Analytics

Alfredo Pirrone, VP Strategic Planning, Cerveceria Regional

Cerveceria Regional is the second largest Venezuelan brewery, holding a 14% share of the beer market. Since 2016 the company set the strategic goal to become a data centred organisation.

What started in 2014 as a traditional BI project based on Data Warehouse technology, rapidly evolved to become a massive initiative, leveraging on self-service analytics.  Becoming data-driven impacted the required tools, the organisation and the company’s culture.

Cerveceria Regional’s data architecture is based on SAP-ERP and its Data Warehouse run on SAP-BW 7.4. In 2015 the company adopted Tableau as graphical analysis and data visualization tool. No Big Data technologies has yet been adopted.

The presentation will distil the experience of becoming data-centred, providing insights and lessons learned:

• Assessing change readiness
• Shaping the organisation
• Impact on governance and culture
• Unexpected impediments and benefits
• Roadmap

AI and IoT for Good

Naser Ali, Head of Solution Marketing, Hitachi Vantara

In this session, Naser Ali shares his experience working in the IoT and AI space; covering complexities, pitfalls, and opportunities to explain why innovation isn’t just good for business — it’s a societal imperative.

Key takeaways include:

• Deeper understanding of what Big Data, IOT, and AI mean at a functional level, not just what brands the buzzwords refer to.
• Detailed understanding of some use-cases, and why solving these is more complex than it seems.
• Not just what it’s for, but who it is for, and how to think about the “business case” or social imperative around it.

Data Management in Manufacturing

Felix Streichert, Data Governance Manufacturing, Robert Bosch GmbH

Digitalization in manufacturing promises a large potential for cost savings, but is faced with a number of challenges, e.g. cost pressure, complex value chain networks, and legacy systems. To introduce data management to this environment requires a step-by-step approach yielding benefits all along the way. We propose a data process life-cycle that allows us to incrementally improve, exploratively analyze, and finally standardize data. Allowing for both limited and full data management in the same framework, while giving a path from one to the other.

• I4.0 and Big Data
• Perpetual Brown Field
• Data Management Approach
• Data Process Life-Cycle

Plenary Keynote: Challenges of Developing an Enterprise Data Marketplace

Rick van der Lans, R20/Consultancy

There is a new kid on the data block: the data marketplace. In a data marketplace business users shop for the right data products. Examples of data products are predefined KPI’s, reports, files, and data services. The data marketplace is a supply-driven architecture in which data products are developed before the business requests them. This is very similar to how most shops operate: products are researched and developed before there is any guarantee that they will be bought. One of the key goals of the data marketplace is to let organizations benefit more from the investment they have made in data over the years. In this Keynote, Rick explains how an enterprise  data marketplace differs from a data warehouse and a data lake. Additionally, the challenges of developing and maintaining a data marketplace are discussed. Because it’s another data delivery system developed to supply business users with the right information at the right time, some incorrectly think it’s the old data warehouse, but with a twist. Rick will address this common misunderstanding, in general the data marketplace extends the capabilities of existing data delivery systems, such as data warehouses, data marts and data lakes.

Business Intelligence & Analytics Keynote: Driving Change by Applying Analytics Enterprise-Wide

Ian Wallis, Head of Data, Analytics & Insight, Defence Infrastructure Organisation

The application of BI and analytics can often be in a specific function, such as sales, marketing, HR or Finance. However, there is a growing awareness that these techniques can be applied to any part of an organisation to drive change and release value. This presentation will explain how the use of BI and analytics can be applied to a range of activities and a centralised team can deploy their skills and experience from one functional activity to another, thereby broadening the capabilities of that team and achieving significant benefits for the organisation.

Enterprise Data Keynote: The Producer, the Consumer, the Owner and the Rest of the World: Governing Big Data

Jan Henderyckx, Managing Partner, Inpuls

Big data governance is not just about making sure that you efficiently use your Hadoop cluster or assuring that you work on the relevant use cases. With the democratization of big data capabilities and the wider access to data, questions arise on the regulatory and ethical compliance of the data usage. Locking all data down is not the answer as we would lose too much value. This presentation focuses on the steps you need to take to get sustainable and compliant value out of your big data.

What delegates will learn from attending the session:

• What is the distinction between information and big data governance
• Catering to the dynamics of data onboarding and usage flows towards policy-based classification and access
• Use case governance vs critical data elements impact of the big data governance requirements on the architecture

Plenary Keynote: Digital Business: Tomorrow is Already Here

Andreas Bitterer, Chief Analytics Evangelist EMEA, SAP

Digital business is about intelligently connecting people, things and businesses. It’s an infinite world of new possibilities for companies to reimagine their business models, the way they work, and how they compete. New technologies like machine learning, the Internet of everything, blockchain, or cloud, etc will transform value chains to enable completely new ways of doing business and our way of life. Hear how leading organizations deliver an innovative customer experience, leveraging the latest technologies, and based on the creative use of a wide variety of information assets.

Is your Company Ready for Self-Service BI?

Ivan Schotsmans

Most companies are still in a traditional warehouse mode, and not ready for self-service BI. , in some departments business users are handling their own data but it’s a silo based approach manipulating and extending trusted data source. For self-service a companywide approach is needed where results are based on data sources.  not about tools, there is no switch to turn traditional into self-service BI. It all starts with an architecture a multipurpose BI environment.   To create a service BI environment and business users make decision based on facts we need to take into account a number of key factors:

• Do we all need to be data scientists?
• Which architecture do we put in place?
• How do we handle data governance?
• How do we guarantee security?
• Did we foresee the necessary change management?

Key Takeaways:

• Architecture for Self-Service BI
• How to translate data into information
• Need for Change management
• Data Correctness

Edge Analytics and Client-Side Machine Learning

Timo Kunz, Data Scientist, Catawiki

The proliferation of Internet of Things (IoT) devices and the corresponding surge in volume of data streams has created a need for processing and analysing data closer to where it is generated: at the edge of the network. In a similar fashion, Machine Learning models are coming to the browser rather than data being sent to a server. A number of powerful tools have become available recently that make this approach now very accessible: the development of sophisticated applications that make efficient use of the user’s hardware capacity and even allow model training on client side is now possible for everyone. This session introduces some recent trends and highlights some free, state-of-the-art tools that can get you started. A demo illustrates a simple implementation.

How to Lower Costs Using IoT Supported by AI

Majken Sander, Independent Consultant   

This session takes delegates through real-life cases: The story from a canteen on how to save money and help the environment; another about automatically meeting relocation based on sensors and algorithms. Also included is a retail store turning down an idea to saving money, because customers expect otherwise, proving that sometimes the smartness of IoT is outsmarted by end users and company optimisation parameters.

How Data Governance Enables Building Self Service BI and Analytics Capabilities

Sachin Dhoble, Head of Data Management, Maersk Line

Data Democratization and empowerment of business users by enabling self-service business intelligence and analytics capabilities has become an increasingly popular trend among global enterprises. This makes Data Governance setup even more critical to ensure that the business users have a common and consistent understanding of data, can trust the data and understand the relationships to be able to utilize the self-service capabilities in informed decision making. This session covers learnings from an initiative for setting up the right Data Governance setup for self-service BI and Analytics in addition to an Agile setup for delivering traditional BI. The approach around how to ensure that Data Governance setup becomes an enabler instead of a hindrance with several check points and approvals, will also be covered. The advantages and challenges of moving towards agile setup for BI delivery will also be discussed.  The attendees will learn:

• Approach for setting up data governance framework as an enabler for self-service BI and Analytics
• What are the critical components in data governance framework essential for the success of self-service and agile BI delivery
• How to build trusted data sources for self-service to ensure correct decision making

Fastest Answer Wins – How Nordea Accelerates Data-Driven Projects

Paul Lynn, Product Engineering Team Lead, Nordea

Today’s institutions are demanding ever quicker project deliveries. However, the initial steps of data exploration and preparation that are essential to successful delivery, are also often the most frustrating and time-consuming. The 80/20 rule is a familiar situation. A key problem is, how to accelerate this initiation project phase. The prize is quicker implementation and releasing resources to the next revenue creating project.

Join Paul Lynn, Product Engineering Team Lead from Nordea Bank AB, the largest bank in Scandinavia and a GSIB, as he discusses using targeted BI & Analytics to

• Address the task of data consolidation; reducing time spent in the data exploration and preparation phase.
• Bring data to Business users. Enabling self-service analytics for data exploration and preparation
• Remove assumptions and ambiguity from the data
• Act as a work mulitpler, spurring innovation within the organization

Three learning points

• How data-driven organisations can improve both adhoc and productionised business analytics
• Steps to bring down the barrier between business and the data
• Re-engineering of data-driven processes to reduce dependency on IT

Tell your Story with Data

Hylke Peek, Consultant BI and Data Analytics, VX Company.

Let the data speak for itself? The data needs some help with this. The visualizing process is getting more complex. There are many tools, different visuals for the same goal and a diverse audience. If you want to communicate effectively using visualizations, you need to know your audience, understand the context and think like a designer.

We know all the cheat sheets for the best visualizations and how to use colours. In this session, we’ll go further. A report is more than a collection of individual – well formatted – visuals. It’s a story you want to tell. Actually, it’s a story you want the data to tell.

• Understand the context and audience
• Design a (set of) report(s) as one story
• Process of designing visualizations

Laying the Foundations Towards a Data-Driven Future

Jon Evans, Equillian & Sarah Whittle, Data Manager, LiveWest

Formed from the recent merger of Knightstone and Devon & Cornwall Housing, LiveWest is embarking on a journey of transformation to become a housing association of the future – an organisation that puts data at the heart of everything it does, from managing its stock as efficiently as possible to delivering the very best service to its tenants.

Developing a strategic approach to data within a very traditional sector, coupled with the added complexity of bringing together two organisations with differing cultures, processes and data landscapes, would present a number of challenges.

In this case study, LiveWest’s Data Manager, Sarah Whittle, will be joined by Jon Evans, the founder of Equillian, to describe the journey so far. Delegates will learn how LiveWest:

• Created a “data vision” to set out its long-term aspirations for data in a way that would excite and enthuse senior stakeholders
• Conducted a maturity assessment to understand the current approach to managing data and how this differs across the merged organisation
• Developed a roadmap for improving its core data management capabilities and laying the foundations towards a data-driven future

Making Data Mainstream: Establishing a Data Function and Selling the Opportunities it Brings to a Commercial Organisation

Amy Balmain, Head of Data Exploitation, Southern Water

For a long time data has been seen as just a by-product of business processes. Now, established and emerging tools and techniques allow organisations to unlock the real value of data to make significant improvements in operational performance. Attendees will learn:

• Context on why a dedicated data team is a must for business
• How to set one up without importing expensive specialist skills or external consultants
• The significance of the commercial opportunities

The Analytics Factory

Jos van Dongen, Tholis Consulting  

Data science, machine learning and AI are hot topics in todays analytics landscape. Many breakthroughs have been made due to advances in algorithms and computing power. Organizations adopt these new technologies and use them to their own advance to improve sales, customer interactions or internal processes. But, they are also facing new challenges when developing and deploying analytics solutions. Not all solutions are easily scalable or can be deployed and run in an automated way, and more often than not point solutions are created using different tools on platforms which are hard to maintain. Moreover, the performance of analytical models degrades over time, requiring a different type of monitoring and maintenance. This session shows how to overcome these challenges and will address the following topics:

• How to industrialize analytical development processes for sustainable results
• Pros and cons of lambda, kappa and other architectures
• How to design, build and scale your analytics factory

Avoiding Data Warehousing Failure - Experiences Building a Logical Data Warehouse

Norbert Eschle, Enterprise Data Architect, Direct Line Group

In 2005, Gartner reported that about 50% of data warehousing projects fail. In 2011, they stated that between 70% and 80% of Business Intelligence projects fail. Online publications estimate a big data project failure rate at up to 85%. Despite all of this, business functions still need an enterprise view of business data.

The high cost and effort of moving and integrating data from a wide variety of data sources seems to be a key contributor to these failures both due to cost and lack of agility.

In this talk, Norbert will describe how his organisation is looking to avoid such pitfalls and challenges by applying alternative approaches and move towards the concept of a logical data warehouse (LDW). In this talk, attendees will learn of:

• The architectural approach taken to building an LDW
• The outline operational and governance capabilities required
• The business benefits from some of the key architecture decisions taken
• The journey to date and pitfalls avoided

Make Insights a Team Sport with Data and AI

Lena Woolf, Senior Technical Staff Member, IBM

Why are enterprises struggling to capture the value of AI? AI is not magic- it requires teamwork. AI is algorithms + data + team.
But data resides in silos & difficult to access. If the data isn’t secure, self-service isn’t a reality. Enterprises need an environment that enables a “fail fast” approach and provides governed access to data. They also need to provide data scientists, application developers and subject matter experts with set of tools to collaboratively and easily work with data and use that data to build, train and deploy models at scale. In this session we will outline best practices for setting up end-to-end AI workflow and enablement team productivity and collaboration.

Audience will find answers to the following questions:
– what can AI do for my business?
– how do I infuse AI into business to drive innovation?
– I hired a team of data scientists – what’s next?
– how can our data scientists gain access to right data?

How do you Build a Data Strategy When the Execs Don't Really Get It

Abdulhameed Al Amer, SVP Data Strategy, Riyad Bank & Chris Bradley, Data Management Advisors 

A key component of any transformational change program is concerned with communication and organization readiness.  There’s no point in spending significant amounts on technology when the organization and culture isn’t prepared for data management change.

This case study describes the drivers behind Riyad Bank’s data transformation program, the steps undertaken, and the tactics employed to move ahead particularly when many of the Executive team didn’t really understand.

Plenary Keynote: Ethics Schmethics: Hype or Hope?

Daragh O Brien, Castlebridge

Information Management is at a tipping point. The tools and technologies we have developed have great potential, but bring with them great risks. This is increasingly recognised by industry leaders, front-line workers, legislators, and Regulators. In this keynote, you will get a whistle-stop tour through how we got here, why it matters, what it means, and what we can do to ensure that how we manage and use information in the 21st century is trusted and trustworthy, and what lessons we need to learn from the past.

Conference Close

Rick van der Lans & Jan Henderyckx

Jumpstart your Enterprise Data Initiatives and Keeping Them on the Right Track

Jan Henderyckx, Managing Partner, Inpuls

You are convinced that data can bring value to your organisation and your company might have already started the data journey.  In reality many of the data lake and big data initiatives fail because there is more to it than just putting large quantities of data in a data lake.  This workshop will cover the key challenges that organisations faces when embarking on the data journey. You will understand what is required for data based value creation and know how to put the relevant building blocks in place to assure that your organisation interacts with the data in the most effective and efficient way.. Combining risk, privacy, security and value creation separately is the way that leads to proper management of the data. You will also learn how to maximise the use of the data by capturing the usage constraints of the data as early as possible.

Attendees will learn:

• What the key building blocks of a data strategy consist of
• Learn how to engage your business and have them take the lead and recognise the value of information.
• Learn how to adapt the organisation to make information centric
• Learn how to establish an information governance organisation
• What kind of information governance styles can be applied
• Which organisational structure as most suited for creating data value
• How to best integrate with an analytics roadmap
• How tooling can support the value delivery
• Learn how to manage speech communities and business vocabularies
• Learn how to align your IT with your information strategy
• Get more value out of your MDM projects
• Learn how to redefine your Business Intelligence architecture
• Learn how to get the benefits of Big Data
• Select the proper Enterprise Information platform to support your information strategy.

New Big Data Storage Technologies: From Hadoop to Graph Databases, and from NoSQL to NewSQL

Rick van der Lans, R20/Consultancy

Big data, analytical database servers, Hadoop, NoSQL, Spark, MapReduce, SQL-on-Hadoop, translytical databases, and appliances are all immensely popular terms in the IT industry today. Due to this avalanche of new developments, it’s becoming harder and harder for organizations to select the right tools. Which technologies are relevant? Are they mature? What are their use cases? Are they worthy replacements for the more traditional SQL products? How should they be incorporated in the existing data warehouse architecture?” These are all valid but difficult to answer questions.

This full-day workshop discusses and explains these new data storage technologies clearly and explains why and how they can be relevant for any organization. Market overviews are presented, strengths and weaknesses are discussed, and guidelines and best practices are discussed. It is intended for anyone who has to stay up to date and implement the new developments, including data warehouse designers, business intelligence experts, database specialists, database experts, consultants, and technology planners.

• Why are traditional database technology not “big” enough?
• How different are Hadoop and NoSQL are from traditional technology?
• How can new and existing technologies such as Hadoop, NoSQL, and NewSQL help develop BI and big data systems?
• Embedding Hadoop technologies in existing BI systems
• Using Spark to boost performance for analytics
• Three NoSQL subcategories: key-value, document, and column-family stores
• Why graph databases are very different from all other systems
• When to use NewSQL or NoSQL for developing transactional systems

Integrating Fast Data, Edge Analytics and Operational Decision Management into Your BI Environment

Jos van Dongen, Principal Consultant, Tholis Consulting

For many years companies have been building data warehouses and data marts supplied by data extracted from traditional on-line transaction processing systems using batch-oriented ETL processing. However over recent years, as the speed of business quickens, we have seen a huge growth in demand for so called ‘fast data’ (also known as streaming data) to be brought into the enterprise for analysis. There are many popular sources of fast data including telemetry in business operations where organisations have often been blind to problems in the past. Also, Internet of Things (IoT) data from smart consumer products and smart industrial equipment. Live clickstream recording on-line behavior as people surf your website is yet another example, as is data generated from pixels and tags embedded in digital marketing. Others include markets data and social media data. All of this is relatively new in the world of analytics. The benefit of analyzing fast data are obvious. It allows organisations to go beyond traditional BI and start analyzing ‘in-flight data’ in real-time before it is stored anywhere. This allows organisations to optimize their business operations, avoid unplanned operational cost, mitigate risk and see opportunities early so that they can become early movers in real-time markets. However, there are issues with this kind of data. It can be huge in volume. It never stops! Schema can change and new schema can appear without notice. Data can be missing, arrive out of sequence or arrive in huge bursts. Also, how do you process it? How do you analyse it? What kinds of analyses are relevant with this kind of data? What technologies do you need? Where do you analyse it? Should it be done at scale in the data centre or in the cloud? Why not at the edge, closer to where the data is being generated? With so much data being generated and much more to come, would pushing analytics into the network not scale better than streaming analytics at the centre? If so what problems does that bring? Also, what about decision management? How do you automate decisions? Can machine learning help? And how do you integrate all this with traditional BI environments?

This session looks at this problem and discusses the following:

• What is fast data?
• Streaming versus realtime versus right time analytics
• Prevention and opportunity – use cases for streaming analytics
• The characteristics of fast data and the challenges that these bring
• How do you process fast data and integrate it with enterprise data?
• Why is master data important in fast data analytics?
• What is time series analysis?
• Approaches to streaming analytics – in-flight analysis versus fast write processing and near real-time post write analysis
• Technologies options for analyzing fast data:
  • Scalable messaging like Kafka
  • Streaming analytics platforms
  • Fast write NoSQL data stores and file systems
  • Hadoop and cloud options
  • In-memory processing e.g. MemSQL, Apache Spark, Apache Flink
  • Coding vs configuring,
  • Build vs buy vs assemble
• Developing prescriptive machine learning models for real-time analysis
• Building streaming analytic application pipelines using ETL tools
• Model development & training – in stream vs in memory
• Model deployment – central execution, edge analytics or both?
• Model management and monitoring in a fast data world
• Combining models & business rules in real time decision management systems
• Architecture – how should it change to integrate fast data with existing analytical systems such as data warehouses and data marts?
• How to integrate fast data with traditional BI tools?
• Getting started – what do you need to do? Do’s and dont’s