Statistical thinking for Data Science and Analysts

Statistics refers to the process of generating conclusions or scientific truths of data that is analyzed and interpreted. Thus, statistical data plays a vital role in decision-making. Statistical inference helps to produce new knowledge about data and considered as one of the pillars of Data Science.

In this Statistical thinking process, you start to use statistical and mathematical ways of thinking, and your knowledge of computer science tools, to perform experimentation and hypothesis testing in data science and create data-driven solutions.

Statisticians perform greater data analysis to attain the role of Data Scientists. Statisticians develop methods for the design and efficient implementation of studies like surveys, randomized experiments and observational studies that helps to generate data and answer real-world questions. Statisticians use this generated data to extract usable information from the data. Data analysis includes methods for describing and visualizing relationships in the data and building statistical models which is essential to make inferences or predictions about questions of interest and provide measures of useful inferences.

Statistical thinking applies to both methods of management and business operations. The need has increased rapidly to utilize the statistical thinking for tangible business improvements. Statistical analysis improves the performance of a data scientists by taking informed action.

The different models of statistical inference include data oriented strategies, sampling models, hypothesis testing, probability models, non-parametric bootstrapping, permutation, statistical modeling, and randomization in the analytical study.

This course helps to understand the broad spectrum of statistical inference and use the information to make informed choices in data analysis.

Why Statistical Thinking?

Statistical thinking for data science is most evident as it predicts future trends from data, obtain supporting evidence for data based decisions, visualizing data, deriving insights, designing data collection and constructing models.

Why Should I learn Statistical Thinking?

The statistical inference is essential for data science to navigate between the set of assumptions and tools to derive conclusions from data. It serves as an integral part of Reengineering, Learning organizations, TQM – Total Quality Management, Self – managed work teams, Six – Sigma and Benchmarking are widely used statistical methods that focus on ways to manage and reduce variation. Learning the models and methods of statistics contributes to reduce variation and provide quality improvement.

What will I learn?

Statistical methods for data science plays an active role for large collections of information from the data.

• Learn to perform Data collection, preparation, analysis, and inference
• Classify data by the key traits of the customer
• Understand the conditional probability based on certain conditions to judge an event
• Gain knowledge about the basics of Linear Regression
• Use data visualization to create graphs of data
• Use Bayesian modeling and inference to study and forecast public opinion
  • Estimate the uncertainty of a population quantity
  • Determine a benchmark value for the population quantity
  • Infer a mechanistic relationship between  the amounts measured in the population
  • Ascertain the impact of a policy
  • Use the probability methods in a population
  • Establish a basis for hypothesis testing and experiment statistical models for data-driven solutions

Prerequisites for this Tutorial:

Line

• Basic Coding and Scripting knowledge
• Working knowledge of R Programming language (SWIRL – Statistics With Interactive R Programming)

Audience for this Tutorial:

• Software Engineers, Developers willing to pursue their career path to Data Science
• Data Analysts from any domain/industry who are interested in entering Tech industry

Course Syllabus:

Lesson 1 – Introduction to Data Science

Data Science is the practice of exploring, obtaining, modeling and interpreting data. It explores topics such as statistics, mathematics, visualization, machine learning, data analysis, and programming.

• Overview of Data Science
• Importance of Data Science
• Role of Data Visualization in Data Science
• Role and importance of Data Scientists

Lesson 2: Statistical Thinking and Data Science

Data Science is a combination of two fields, statistics/mathematics, and computer science. Statistical Inference is the process of making conclusions using data that is subject to random variation. Probability plays a vital role in statistical inference. Probability helps us to answer questions that naturally arise when analyzing experimental data

Class 1:

• More about Statistical Thinking for Data Science
• Simple Visualization using numerical data
• Summary Statistics using numerical data
• Numerical Data Association

Class 2:

• Probability Introduction   
• Introduction to Statistical Inference
• Identifying Confidence Intervals using Statistical Inference
• Performing Significance tests Statistical Inference

Class 3:

• Association and Dependence
• Data Collection Sampling

Lesson 3: Statistics and Probability

This chapter helps you learn the essential concepts in Statistics and Probability.

Bayes formula is useful for determining conditional probability. The formula provides a way to revise existing predictions or theories given new or additional evidence. This lesson also provides an overview of statistical methods for association studies that include analyses, testing, inference and identifying the missing data.

Regression analysis allows you to estimate the relationship between a response variable to a set of predictor variables.  Linear regression in statistics refers to the modeling method which identifies the relationship between a scalar dependent variable and another independent variable.

Class 1:

• Conditional Probability
• Bayes' Formula

Class 2:

• Studying Association using Two-way Table
• Studying Association using Chi-square Test of Independence
• Studying Association using One-way Analysis of Variance

 Class 3:

• Regression Analysis
• Introduction to Linear Regression
• Special Regression Models

Lesson 4: Exploratory Data Analysis and Visualization

Data recorded in experiments or surveys is presented using a statistical data graph. There are several types of graphs in the Statistical data graph. The different types of graphs used in Statistical data graph include Box plot, Stem and leaf plot, Frequency Polygon, Line Graph, Bar Graph, Scatter Plot, Histogram, Pictograph, Pie chart, Line plot and Map chart. A graph is another method of representing the statistical data in the visual form (data visualization). The use of a specific type of graph from the available different types of the graph is dependent on the data, and the purpose of the graph plotted.  

Class 1:

• Data Graphs
• Use Data  graphs of fitted models
• Use graphs to check fitted models
• Overview of Graphics Principles

Class 2:

• Data visualization with examples
• Decision Making Process of Data Visualization
• Dashboards
• Dashboards with Examples

Lesson 5: All About Bayesian Modeling

This lesson talks about the Bayesian Modeling which is a rule about the ‘language’ of probabilities that is applicable in any analysis for describing random variables. Bayesian methods serve as a whole paradigm for both statistical inference and decision making under uncertainty.  Bayesian methods contain particular cases that are more often used and solves many of the difficult problems faced by using traditional statistical analysis.

In particular, Bayesian methods allow you to incorporate scientific hypothesis in the analysis and apply to problems whose structure is too complex for any conventional methods to handle.

Class 1:

• Introduction to Bayesian Modeling
• Probability Calibration
• Probability As Measurement of Uncertainty
• Bayesian Inference

Class 2:

• Bayesian Hierarchical Modeling in Practice
• Bayesian modeling for Big data
• Introduction to Business Applications in Bayesian Statistics