Foundations of Data Science Project - Diabetes Analysis

---

Context

---

Diabetes is one of the most frequent diseases worldwide and the number of diabetic patients are growing over the years. The main cause of diabetes remains unknown, yet scientists believe that both genetic factors and environmental lifestyle play a major role in diabetes.

A few years ago research was done on a tribe in America which is called the Pima tribe. In this tribe, it was found that the ladies are prone to diabetes very early. Several constraints were placed on the selection of these instances from a larger database. In particular, all patients were females at least 21 years old of Pima tribe.

---

Objective

---

Here, we are analyzing different aspects of Diabetes in the Pima Diabetes Analysis by doing Exploratory Data Analysis.

---

Data Dictionary

---

The dataset has the following information:

• Pregnancies: Number of times pregnant
• Glucose: Plasma glucose concentration over 2 hours in an oral glucose tolerance test
• BloodPressure: Diastolic blood pressure (mm Hg)
• SkinThickness: Triceps skin fold thickness (mm)
• Insulin: 2-Hour serum insulin (mu U/ml)
• BMI: Body mass index (weight in kg/(height in m)^2)
• DiabetesPedigreeFunction: A function that scores the likelihood of diabetes based on family history.
• Age: Age in years
• Outcome: Class variable (0: a person is not diabetic or 1: a person is diabetic)

Q 1: Import the necessary libraries and briefly explain the use of each library (3 Marks)

# remove _____ & write the appropriate library name

import numpy as np
import pandas as pd

import seaborn as sns
import matplotlib.pyplot as plt
%matplotlib inline

Write your Answer here:

Ans 1:NumPy: NumPy is also called as Numerical Python and this library is used to perform wide variety of Mathematical Operations on the datasets.It supports large multi-dimensional arrays for data handling. Pandas: Pandas are useful for Data cleaning and analyzing datasets.They can also be used to import data efficiently from different file formats like csv,json etc.They run on top of Numpy. matplotlib: This is Data visualization library for plotting basic graphs line, bar graphs etc. This import python plot(.pyplot) to visualize the graphs. Seaborn: Seaborn is an advanced data visualization library and is based on matlplotlib. It provides an interface for statistical graphs and drawings. It enables us to visualize sophisticated graphs like box plot, heatmaps, pairplots etc.

Q 2: Read the given dataset (1 Mark)

# remove _____ & write the appropriate function name
pima=pd.read_csv('diabetes.csv')

Q3. Show the last 10 records of the dataset. How many columns are there? (1 Mark)

# remove ______ and write the appropriate number in the function

pima.tail(10)

	Pregnancies	Glucose	BloodPressure	SkinThickness	Insulin	BMI	DiabetesPedigreeFunction	Age	Outcome
758	1	106	76	20	79	37.5	0.197	26	0
759	6	190	92	20	79	35.5	0.278	66	1
760	2	88	58	26	16	28.4	0.766	22	0
761	9	170	74	31	79	44.0	0.403	43	1
762	9	89	62	20	79	22.5	0.142	33	0
763	10	101	76	48	180	32.9	0.171	63	0
764	2	122	70	27	79	36.8	0.340	27	0
765	5	121	72	23	112	26.2	0.245	30	0
766	1	126	60	20	79	30.1	0.349	47	1
767	1	93	70	31	79	30.4	0.315	23	0

Write your Answer here:

Ans 3:Total Columns:9

Q4. Show the first 10 records of the dataset (1 Mark)

# remove _____ & write the appropriate function name and the number of rows to get in the output

pima.head(10)

	Pregnancies	Glucose	BloodPressure	SkinThickness	Insulin	BMI	DiabetesPedigreeFunction	Age	Outcome
0	6	148	72	35	79	33.600000	0.627	50	1
1	1	85	66	29	79	26.600000	0.351	31	0
2	8	183	64	20	79	23.300000	0.672	32	1
3	1	89	66	23	94	28.100000	0.167	21	0
4	0	137	40	35	168	43.100000	2.288	33	1
5	5	116	74	20	79	25.600000	0.201	30	0
6	3	78	50	32	88	31.000000	0.248	26	1
7	10	115	69	20	79	35.300000	0.134	29	0
8	2	197	70	45	543	30.500000	0.158	53	1
9	8	125	96	20	79	31.992578	0.232	54	1

Q5. What do you understand by the dimension of the dataset? Find the dimension of the pima dataframe. (1 Mark)

# remove _____ & write the appropriate function name

pima.shape

(768, 9)

Write your Answer here:

Ans 5:The shape of the dataset is a tuple of two elements. Number of rows in the dataset is shown by the first element i.e 768 Number of columns in the dataset is shown by the second element i.e 9

Q6. What do you understand by the size of the dataset? Find the size of the pima dataframe. (1 Mark)

# remove _____ & write the appropriate function name

pima.size

6912

Write your Answer here:

Ans 6:The total number of elements in the data (rows * columns) is defined as the Size of the dataset which is 6912.

Q7. What are the data types of all the variables in the data set? (2 Marks)

Hint: Use the info() function to get all the information about the dataset.

# remove _____ & write the appropriate function name

pima.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 768 entries, 0 to 767
Data columns (total 9 columns):
 #   Column                    Non-Null Count  Dtype  
---  ------                    --------------  -----  
 0   Pregnancies               768 non-null    int64  
 1   Glucose                   768 non-null    int64  
 2   BloodPressure             768 non-null    int64  
 3   SkinThickness             768 non-null    int64  
 4   Insulin                   768 non-null    int64  
 5   BMI                       768 non-null    float64
 6   DiabetesPedigreeFunction  768 non-null    float64
 7   Age                       768 non-null    int64  
 8   Outcome                   768 non-null    int64  
dtypes: float64(2), int64(7)
memory usage: 54.1 KB

Write your Answer here:

Ans 7:There are in total two datatypes in the given dataset.They are int64 and float64.(Both are numerical variables). 1. integer(int64) is a number without a decimal point.In the given dataset, there are 7 columns: Pregnencies,Glucose,BloodPressure,SkinThickness,Insulin,Age,Outcome (The values of Outcome column are 0 and 1) 2. Float(float64) is a number with a decimal point. There are 2 columns with float dtype:BMI,DiabetesPedigreeFunction.

Q8. What do we mean by missing values? Are there any missing values in the pima dataframe? (2 Marks)

# remove _____ & write the appropriate function name

pima.isnull().values.any()

False

Write your Answer here:

Ans 8:Missing Values tells us about the missing information in the dataset i.e if there are any blank/Null values in the cells of the dataset. The Pima Diabetis dataset has no missing values since the result of the above code is False.

Q9. What do the summary statistics of the data represent? Find the summary statistics for all variables except 'Outcome' in the pima data. Take one column/variable from the output table and explain all its statistical measures. (3 Marks)

# remove _____ & write the appropriate function name

pima.iloc[:,0:8].describe()

	Pregnancies	Glucose	BloodPressure	SkinThickness	Insulin	BMI	DiabetesPedigreeFunction	Age
count	768.000000	768.000000	768.000000	768.000000	768.000000	768.000000	768.000000	768.000000
mean	3.845052	121.675781	72.250000	26.447917	118.270833	32.450805	0.471876	33.240885
std	3.369578	30.436252	12.117203	9.733872	93.243829	6.875374	0.331329	11.760232
min	0.000000	44.000000	24.000000	7.000000	14.000000	18.200000	0.078000	21.000000
25%	1.000000	99.750000	64.000000	20.000000	79.000000	27.500000	0.243750	24.000000
50%	3.000000	117.000000	72.000000	23.000000	79.000000	32.000000	0.372500	29.000000
75%	6.000000	140.250000	80.000000	32.000000	127.250000	36.600000	0.626250	41.000000
max	17.000000	199.000000	122.000000	99.000000	846.000000	67.100000	2.420000	81.000000

Write your Answer here:

Ans 9:'Age' column: 1. The minimum and maximum age of Diabetes patients are 21 and 81 respectively. 2. The average age of a Diabetic patient i.e mean of the Age is 33 3. The standard deviation tells where the data is clustered or scattered around the mean. Here the std of Age is 11. 4. The count function gives the number of entries the Age column has. It has in total 768 entries. 5. Median-The median age of the Diabetic patients is 29yrs (meaning- 50 percentile of patients are <= age 29). 6.Lower quartile(25%) from the data is 24 which means 25 percentile of the patients are under the age 24. 7.Upper quartile(75%) from the data is 41 which means 75 percentile of the patients are under the age 41. From the above data, the mean is greater than median which infers that the data is skewed towards right.

Q 10. Plot the distribution plot for the variable 'BloodPressure'. Write detailed observations from the plot. (2 Marks)

# remove _____ & write the appropriate library name

sns.displot(pima['BloodPressure'], kind='kde')
plt.show()

Write your Answer here:

Ans 10:Most of the values are between 60 and 90 having peak value at 72.9.
       The data almost looks symmetrical with one peak. This is an Unimodal distribution.

Q 11. What is the 'BMI' of the person having the highest 'Glucose'? (1 Mark)

# remove _____ & write the appropriate function name

pima[pima['Glucose']==pima['Glucose'].max()]['BMI']

661    42.9
Name: BMI, dtype: float64

Write your Answer here:

Ans 11:The BMI of the patient with highest Glucose level of 661 is 42.9.

Q12.

12.1 What is the mean of the variable 'BMI'?

12.2 What is the median of the variable 'BMI'?

12.3 What is the mode of the variable 'BMI'?

12.4 Are the three measures of central tendency equal?

(3 Marks)

# remove _____ & write the appropriate function name

m1 = pima['BMI'].mean()  # mean
print(m1)
m2 = pima['BMI'].median()  # median
print(m2)
m3 = pima['BMI'].mode()[0]  # mode
print(m3)

32.45080515543617
32.0
32.0

Write your Answer here:

Ans 12:The mean, median and mode of BMI are 32.45,32,32 From the above data, we can infer that the median and mode are equal whereas the mean is very slightly greater than median and mode.

Q13. How many women's 'Glucose' levels are above the mean level of 'Glucose'? (1 Mark)

# remove _____ & write the appropriate function name

pima[pima['Glucose']>pima['Glucose'].mean()].shape[0]

343

Write your Answer here:

Ans 13:The mean Glucose level is 121.67. There are 343 women whose are Glucose levels are above the mean level which is approximately 44%.

Q14. How many women have their 'BloodPressure' equal to the median of 'BloodPressure' and their 'BMI' less than the median of 'BMI'? (2 Marks)

# remove _____ & write the appropriate column name

pima[(pima['BloodPressure']==pima['BloodPressure'].median()) & (pima['BMI']<pima['BMI'].median())].shape

(22, 9)

Write your Answer here:

Ans 14:There are 22 women whose BloodPressure equals to median BloodPressure and BMI < median BMI.

Q15. Create a pairplot for the variables 'Glucose', 'SkinThickness', and 'DiabetesPedigreeFunction'. Write your observations from the plot. (4 Marks)

# remove _____ & write the appropriate function name

sns.pairplot(data=pima,vars=['Glucose', 'SkinThickness', 'DiabetesPedigreeFunction'], hue='Outcome')
plt.show()

Write your Answer here:

Ans 15:Between glucose levels (x-axis) and diabetes pedigree function (y-axis), diabetic patients have more glucose level which can be seen on the graph with orange dots towards the right of the graph. However, the diabetes pedigree function is distributed all across. Between Skin Thickness (y-axis) and glucose levels (x-axis), skin thickness values are distributed all across which means between diabetic and non-diabetic patients we can't say one grou has significantly thicker skin than other group.

Q16. Plot the scatterplot between 'Glucose' and 'Insulin'. Write your observations from the plot. (2 Marks)

# remove _____ & write the appropriate function name

sns.scatterplot(x='Glucose',y='Insulin',data=pima)
plt.show()

Write your Answer here:

Ans 16:Majority of people with low and high glucose levels are populated in the lower part of the graph (insulin<300). Its interesting to see some people with glucose level >140 has high insulin level and are populated at the top right of the graph with couple outliers.

Q 17. Plot the boxplot for the 'Age' variable. Are there outliers? (2 Marks)

# remove _____ & write the appropriate function and column name 

plt.boxplot(pima['Age'])

plt.title('Boxplot of Age')
plt.ylabel('Age')
plt.show()

Write your Answer here:

Ans 17:Yes, there are outliers on the data (age>65).

Q18. Plot histograms for the 'Age' variable to understand the number of women in different age groups given whether they have diabetes or not. Explain both histograms and compare them. (3 Marks)

# remove _____ & write the appropriate function and column name

plt.hist(pima[pima['Outcome']==1]['Age'], bins = 5)
plt.title('Distribution of Age for Women who has Diabetes')
plt.xlabel('Age')
plt.ylabel('Frequency')
plt.show()

# remove _____ & write the appropriate function and column name

plt.hist(pima[pima['Outcome']==0]['Age'], bins = 5)
plt.title('Distribution of Age for Women who do not have Diabetes')
plt.xlabel('Age')
plt.ylabel('Frequency')
plt.show()
pima[(pima['Outcome']==1) & (pima['Age']<=40)].shape

(166, 9)

Ans 18: Among the women with diabetes, 166 women are under age 40 which is 62% of the total diabetic patients. This tells us that more than half of patients are at high risk of diabetes or have diabetes within age 40.

Among the non-diabetic people, majority of the women are under age 30.

Q 19. What is the Interquartile Range of all the variables? Why is this used? Which plot visualizes the same? (2 Marks)

# remove _____ & write the appropriate variable name

Q1 = pima.quantile(0.25)
Q3 = pima.quantile(0.75)
IQR = Q3 - Q1
print(IQR)

Pregnancies                  5.0000
Glucose                     40.5000
BloodPressure               16.0000
SkinThickness               12.0000
Insulin                     48.2500
BMI                          9.1000
DiabetesPedigreeFunction     0.3825
Age                         17.0000
Outcome                      1.0000
dtype: float64

Write your Answer here:

Ans 19:The above are the Interquartile Range for each variable. The range of IQR is between Q1 & Q3. IQR is used to know the dispersion of the data in a dataset (i.e. how widely the data is spread across). It also helps us to identify outliers in the data. Boxplot can be used to visualize quartiles and outliers in the data.

Q 20. Find and visualize the correlation matrix. Write your observations from the plot. (3 Marks)

# remove _____ & write the appropriate function name and run the code

corr_matrix = pima.iloc[:,0:8].corr()

corr_matrix

	Pregnancies	Glucose	BloodPressure	SkinThickness	Insulin	BMI	DiabetesPedigreeFunction	Age
Pregnancies	1.000000	0.128022	0.208987	0.009393	-0.018780	0.021546	-0.033523	0.544341
Glucose	0.128022	1.000000	0.219765	0.158060	0.396137	0.231464	0.137158	0.266673
BloodPressure	0.208987	0.219765	1.000000	0.130403	0.010492	0.281222	0.000471	0.326791
SkinThickness	0.009393	0.158060	0.130403	1.000000	0.245410	0.532552	0.157196	0.020582
Insulin	-0.018780	0.396137	0.010492	0.245410	1.000000	0.189919	0.158243	0.037676
BMI	0.021546	0.231464	0.281222	0.532552	0.189919	1.000000	0.153508	0.025748
DiabetesPedigreeFunction	-0.033523	0.137158	0.000471	0.157196	0.158243	0.153508	1.000000	0.033561
Age	0.544341	0.266673	0.326791	0.020582	0.037676	0.025748	0.033561	1.000000

# remove _____ & write the appropriate function name

plt.figure(figsize=(8,8))
sns.heatmap(corr_matrix, annot = True)

# display the plot
plt.show()

Write your Answer here:

Ans 20: Age and Pregnancies are positively correlated moderately with correlaion coefficient of 0.54 SkinThickness and BMI are positively correlated moderately with correlaion coefficient of 0.53. There are variables which have weak negative correlation with other variables such as Pregnancies vs DiabetesPedigreeFunction,Insulin vs Pregnancies etc. From the above data, we can infer that there is no strong correlation between the variables with correlation coefficient >0.7