Docsity
Docsity

Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity


Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan


Guidelines and tips
Guidelines and tips

introduction to data science, Schemes and Mind Maps of Basics of Data Warehousing

This document introduces the concept of data science and its importance in today's world. It discusses the various applications of data analytics and the 5 Vs of big data. It also covers the data science life cycle and the importance of data preparation. the importance of Python in data science and introduces the Pandas library. It also covers the various operations that can be performed on data using Pandas. the problems associated with dirty data and the importance of data preprocessing.

Typology: Schemes and Mind Maps

2021/2022

Available from 06/25/2023

parth-pethkar
parth-pethkar 🇮🇳

6 documents

1 / 56

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
UNIT - I
Introduction to Data Science
SCHOOL OF mechanical EGINEERING AND TECHNOLOGY
Data Science
T. Y. BTECH
pf3
pf4
pf5
pf8
pf9
pfa
pfd
pfe
pff
pf12
pf13
pf14
pf15
pf16
pf17
pf18
pf19
pf1a
pf1b
pf1c
pf1d
pf1e
pf1f
pf20
pf21
pf22
pf23
pf24
pf25
pf26
pf27
pf28
pf29
pf2a
pf2b
pf2c
pf2d
pf2e
pf2f
pf30
pf31
pf32
pf33
pf34
pf35
pf36
pf37
pf38

Partial preview of the text

Download introduction to data science and more Schemes and Mind Maps Basics of Data Warehousing in PDF only on Docsity!

UNIT - I

Introduction to Data Science

SCHOOL OF mechanical EGINEERING AND TECHNOLOGY

Data Science

T. Y. BTECH

Data Science: Why all the Excitement?

Exciting new effective applications of data analytics e.g., Google Flu Trends: Detecting outbreaks two weeks ahead of CDC data New models are estimating which cities are most at risk for spread of the Ebola virus. Prediction model is built on Various data sources, types and analysis.

Graph Data 4 Lots of interesting data has a graph structure:

  • (^) Social networks
  • (^) Communication networks
  • (^) Computer Networks
  • (^) Road networks
  • (^) Citations
  • (^) Collaborations/Relationships
  • (^) … Some of these graphs can get quite large (e.g., Facebook

user graph)

What can you do with the data? 5

Crowdsourcing + physical modeling + sensing + data assimilation

to produce:

From Alex Bayen, UCB

DATA SCIENCE – WHAT IS IT?

Data Science – A Definition 9 Data Science is the science which uses computer science, statistics and machine learning, visualization and human-computer interactions to collect, clean, integrate, analyze, visualize, interact with data to create data products.

Jeff Hammerbacher’s Model 11

  1. Identify problem
  2. Instrument data sources
  3. Collect data
  4. Prepare data (integrate, transform, clean, filter, aggregate)
  5. Build model
  6. Evaluate model
  7. Communicate results

Data Scientist’s Practice Digging Around in Data Hypothesize Model Large Scale Exploitation Evaluate Interpret Clean, prep

Data Science: Getting Value out of Data

Data Science: Getting Value out of Data

Data Science: Getting Value out of Data

Why the Increased Interest in Data Science?

Applications

  • (^) Climate change and weather
  • (^) Traffic control
  • (^) Agriculture
  • (^) Personalised healthcare
  • (^) Twitter data analysis
  • (^) Facebook information links
  • (^) Pollution and Weather

Contrast: Databases Databases Data Science Data Value “Precious” “Cheap” Data Volume Modest Massive Examples Bank records, Personnel records, Census, Medical records Online clicks, GPS logs, Tweets, Building sensor readings Priorities Consistency, Error recovery, Auditability Speed, Availability, Query richness Structured Strongly (Schema) Weakly or none (Text) Properties Transactions, ACID* CAP* theorem (2/3), eventual consistency Realizations SQL NoSQL: MongoDB, CouchDB, Hbase, Cassandra,… ACID = Atomicity, Consistency, Isolation and Durability CAP = Consistency, Availability, Partition Tolerance