Data Science with Python: Step-by-Step Guide & Transcript

[00:09] the most important career skills for the AI era. The global big data analytics market is expected to reach around 924.3 billion dollars by 2032. And the data

[00:21] engineering market in India is expected to grow at 35% CAGR from 2025 to 2030. This means that companies are not just looking for people who can collect data,

[00:33] who can understand data, analyze it, visualize it, use AI tools, and turn insights into real business decisions. Because today every company has data, but the real career opportunity belongs to those who know how to use it. With

[00:48] today's session on AI-powered data we will understand how data analytics helps businesses collect, clean, analyze, visualize, and interpret data to make smarter decisions. We will begin

[01:02] understand why data has become one of the most valuable assets for every organization. Next, we will explore how data is collected, cleaned, and prepared before analysis. Because raw data is often incomplete, messy, duplicated, or

[01:16] inconsistent. So, before we find insights, we first need to make the data reliable. And after that, we will move into important tools and concepts like Excel, SQL, Python, NumPy, Pandas, and data visualization. Next, we will

[01:29] explore data visualization, where we are going to convert numbers into meaningful charts, dashboards, and reports. We will also learn how tools like Tableau, Power BI, Matplotlib, and other visualization tools help us present insights clearly

[01:44] so that businesses can understand what is happening and what action could be statistical analysis and machine learning concepts like regression, clustering, and forecasting. These concepts help businesses identify

[01:57] trends, predict outcomes, group similar data points, and make data-driven decisions. We will also learn about data ethics because working with data is just not about data analysis. It is actually about using data responsibly, protecting

[02:10] privacy, avoiding bias, and making fair decisions. And finally, we are going to explore some of the advanced concepts. Now, before you begin our session, just a quick info, guys. Simply Learn has got professional certificate program in data

[02:23] analytics, generative AI, and adaptive systems in collaboration with I Hub program actually helps you build real-world skills in data analytics, generative AI, agentic AI, and adaptive systems through live online classes, 180

[02:37] plus hours of structured learning, 40 plus exercises, 15 plus industry projects, mentor support, and capstone project. You will also gain hands-on experience with tools like Excel, SQL, Python, R, Tableau, Power BI, AWS Glue,

[02:51] Microsoft Azure, and many more amazing tools. This program also includes masterclasses, career support, resume reviews, profile optimization, mock interviews, and job assist plus to help learners prepare for data-driven roles.

[03:04] just understand data, but you will also understand how to turn data into insights. And also, how to turn those insights into decisions and decisions into business impact. Now, before you move ahead, just a short quiz to test

[03:18] your knowledge. What is the main goal of data analytics? And your options are to store data without using it, to convert raw data into useful insights, to delete business records, or to avoid decision-making. Please mention your

[03:30] answers in the comment section below. So, today we will start with what data science is, the basics of data science, what all we need to cover. Practically, we will be starting working with the advanced libraries of Python,

[03:43] Uh you know, where we concept on numerical Python, then we move on to the Pandas library. Then we will move on to data visualization library where we would be covering matplotlib, seaborn, as well as the plotly library.

[03:58] as well as the plotly library. Another important aspect of data science is that no data science is complete without the concepts of statistics and maths. Not very deep concepts of maths, but definitely some concepts of

[04:12] statistics as part of data science. Then we will move ahead and try to understand what probability distribution is, how they really help in analysis of the data. Move on to the advanced statistics part of it which refers to inferential

[04:27] statistics. Data wrangling and feature engineering. These two are the last points of this particular course where we would be practic- working a lot of data sense, how we clean it, refine it, and prepare it finally to be well to be

[04:43] used into the model. So, we would be practically covering and dealing with a lot of real-time data like data imputation, scaling, binning, and imputation, scaling, binning, and grouping of operations. Right? So, these

[04:55] are the course components. So, that is why again I am requesting you and suggesting you to go through your basic Python because that would be required.

[05:07] And as already discussed by the LSM that you already have your LMS, the learning management system where you get your hands-on exercises to practice and gain knowledge. Course and projects as well as e-books uh for reference.

[05:21] Next question is, do you think that data exist only for today or it always existed? How did it exist, Ishan? How did it exist? How did it exist? Even if we talk about always. Yes, Chaitra, always, but how? Just explain, elaborate

[05:37] that point. In raw form, maybe. What kind of raw form, Vaibhav? So, you just think of ancient man. How did the ancient man, uh, you know, store data when it start communicating accounts and banking

[05:52] system? Uh, we still have accounts in banking system, Nigam. I'm talking about, you know, uh, you know, uh, data existing since the evo- evolution of human mankind, you know? So, when it

[06:07] was ancient man, you know, where where we never even had houses, we used to kill animals, you know, cook on fire and everything. And then, we started storing information. Some kind of language was developed in terms of sign language,

[06:21] symbols, and everything was written on maybe on a bark or leaves or on stones, right? Right? We have always been storing the data or whatever is preferable to us, whatever we want to store. We've always

[06:34] whatever we want to store. We've always been storing those in terms of walls or that? And as human, uh, you know, civilization developed, right? The ma- civilized houses, more, uh, you know,

[06:51] like we understand Harappa civilization, Indus Valley civilization, right? We started living in houses with other things. So, the the data also started getting evolved, our education system getting evolved. And now we started

[07:05] writing on the paper, you know, keeping all the accounts, your name, records started keeping. We started keeping into files. Right? We've always kept that data in terms of files. Yes or no?

[07:20] Right? And after the evolution of computers, that's around, you know, the computers, that's around, you know, the 1990 80s and 90s, you know, that the data started getting stored in terms of our computer system. And of course, with

[07:34] the evolution of internet, the whole world now converges. So, since it has been 60s, uh, you know, 7 8 decades to this whole uh, revolution. So, we only understand digital data, right? Today, the data

[07:50] that we talk about about analysis of the data or the science related to it, that data we understand it is in digital in form of digits, stored in terms of

[08:02] memory. That is why you see a big, you know, noise now about battery system, semiconductor devices, you know? Because these are the lifelines. If this hardware is not there, then the hard then the, you know, data also does not

[08:17] make sense. You know, you you you might have heard about this in the AI summit that people are talking about data centers and the energy and the power to need to store that data, right? That is why there is this big burst because now

[08:31] why there is this big burst because now data is the new oil of the whole world. Yes, do you all agree? Whoever has power to the data, whether it's China, America, you know, everybody wants to be a big player. That is where

[08:46] the, you know, that country or that will will have the whole power, right or wrong? wrong? Are you all getting this point?

[09:03] right? So, you only understand digital data. That is why it was important for has always been part of our civilization, right? But today, we are living in the world of digital data, whether it's in mobile phones, whether

[09:18] it's internet. I mean, without internet, there is no lifeline. We cannot even breathe without our mobile phone, you know? Any payment that we want to do, anything that we want to study, anything that we want to do, you know, without

[09:31] that we want to do, you know, without mobile phone, we are helpless. Agreed? Do you all agree? So, when we talk about data, data is nothing but facts and figures, observations, measurements that we take. And now we want to do analysis

[09:48] on this data. Why do we want to do analysis of this data now? Right? The numbers, the fact elements. Tell me. Why? Why do we want to do the analysis of this data? Quickly, tell me, learners. Why? Why is it important to do

[10:04] to do the analysis? Tell me. Why is there so much of buzz around this data? To fetch meaningful information from raw data. So, why do we want this What do we want to do with this meaningful information?

[10:19] Now, I'm going to give you one practical example where you will understand the example where you will understand the use of data science completely. So, data science for business is also known as the DIKW

[10:35] model, where D stands for data, I stands for information, K stands for knowledge, and W stands for wisdom. Now, what does each term mean? Please try to understand. So, data is nothing but raw facts and

[10:51] numbers. So, for example, if I'm talking about sales of a car, right? So, that is the data for a particular company. When I process that data, that becomes my information. For example, what has

[11:07] been the maximum and the minimum sales in the last 1 year. So, maybe the dark blue points are referring to the maximum sales, and the green one is referring to the minimum sales. Now, all the months which has sales has happened.

[11:24] Getting my point? So, again, I'm repeating, data is facts and raw facts and figures. Information is processed data. Now, knowledge means something which has been occurred over the year years, you

[11:39] know, not based on one particular data, but over. So, now I have data for the past 5 years. You know, and now I'm looking at the data for the past 5 years. And what do I observe that maximum sales happen

[11:55] during in the month of my festivities, maybe during my Navaratras or during the financial [clears throat] year of end. This is what my observation is. And minimum sales happens in the month of June and July.

[12:08] Clear? So, this is the insight or the pattern that I have analyzed. And based on these insights, I will have a wisdom that definitely, you know, now what we can do is that we can, you know, now we can

[12:23] give our observation to the, you know, sales team or the marketing team that, you know, maybe we can have more offers during the Navaratras or the festival times, or the Christmas times, right? So, that our

[12:38] sales can have can have more increase or bumper offers so that it can actually impact our business. Getting my point? So, are you now understanding how data will actually

[12:53] have the impact or the, you know, what you were saying were all all of you were correct, but you were not able to relate the term. So, how actual data really has impact on the business by understanding, analyzing the patterns.

[13:09] analyzing the patterns. Right? And this is known as the DIKW model. Clear? Any questions, any doubt till here? Yes, learners.

[13:23] One more example, one more practical example from the New York Times story that, you know, Hurricane Francis, you know, we we see these things happening around us. Hurricane Francis was on its

[13:37] way on the Florida Atlantic coast. On Florida, there was a hurricane which was about to hit, you know, the Florida. And it was Linda you know, the Florida. And it was Linda M. Dillman, Walmart's CIO, the chief

[13:50] information officer. She thought that, you know, one of the hurricane had struck Hurricane Charley Charley had struck several weeks earlier, right? So, what did she think? She thought

[14:04] that, you know, let's try to analyze the data where Hurricane Charley struck several weeks before and try to understand the history of the shoppers, understand the history of the shoppers, right? Because we want a good impact of

[14:19] on our sales, right? So, what do you think? You know, now these trends are quite popular that a a particular hurricane is about to hit, what would be the items that you would like to buy and keep it for yourself? Like when you move

[14:33] to any Walmart or any kind of, you know, Smart Bazaar or Dmart, You know, if I say, you know, uh some natural disaster is about to occur. How do you want to go about it? What do

[14:47] you want all you want to buy? Tell me. buying? Everybody wants to buy basic groceries,

[15:01] fruits, vegetables so that they have sufficient to eat. So, you know, there that could have been and maybe, you know, this is a little older times, uh you know, you know, story 2004 so maybe maybe

[15:15] people would like to watch some kind of a movie CD which was more hit and they would like to buy it that buy more of such things, right? Maybe a flashlight such things, right? Maybe a flashlight or even a power bank would work.

[15:32] the data and found that the stores would indeed and found that the stores would indeed need certain products and not just usual need certain products and not just usual flat flashlights. And from the past, it

[15:44] the the top-selling item was strawberry Pop-Tarts increase in sales strawberry Pop-Tarts increase in sales like seven times their normal sales rate ahead of the hurricane. So, Miss Tillman confessed in an interview that the

[15:58] confessed in an interview that the pre-hurricane top-selling item was a Getting my point? Are you now understanding the power of data and its analysis? It is impacting today's business.

[16:14] Right? So, lesson number two is all about basics of data science. So, what understood? Data science is a multi-disciplinary

[16:27] field that uses scientific methods, right? So, we would be definitely using certain statistics, certain styles, you know, scientific research methods to

[16:39] know, scientific research methods to conclude about our results, processes, algorithms, systems to define, you know, to derive meaningful insights and from to derive meaningful insights and from structured as well as unstructured data.

[16:54] Do we understand the term structured and unstructured data? structured and unstructured data? Structured data means something more

[17:08] Structured data means something more related to in the form of table, Excel file, maybe, or, you know, SQL that you have done. Unstructured data is in terms of audios, videos, log files.

[17:22] videos, log files. Getting my point? me? Come on.

[17:39] science that will help us to give conclusion. So, using a search engine or making a purchase on Amazon provides valuable data to data science driven software system operating in the background.

[17:54] So, data science emerges from a combination of expertise, scientific methodologies, and technology. It's not only about data, but it is also about mathematical and statistics model that

[18:09] we would be covering up in this course. How we would be using it to analyze it, and different scientific tools and methods. So, there are a lot of tools available, different libraries that are available.

[18:22] You know, but in this particular course, we are trying to be using the Python we are trying to be using the Python language to do the analysis or you know, to find the conclusions from the data.

[18:34] Clear? Is this point getting clear to everybody? So, one of the applications of data science in healthcare, which nobody was able to tell me is that we all are using

[18:47] able to tell me is that we all are using smartwatches. my BP, the heart rate? Right? What kind of stress I have, the

[18:59] number of counts I have done. Then it also advises you advises you to go and take an activity, drink some water, it's been a long time. Do we see that you know, things happening through the smart watch.

[19:19] from everybody? Yes or no? Something which is practically that we see around which we are using. Kishan, Khushi Are we using it or not? Pragna, Hurnesh, Pooja

[19:34] Lokesh, Sachin, Sahana, Sahil Shrikant, Siddharth, Shristi, Shreyash, Shreya Tanisha, are you all there? Gagan, Nishan

[19:47] Yeah. So, if you are wearing that smart watch, right? And we transfer the data, the data gets transferred to the different servers. And it is the enterprise server which analyzes the data and you know tells me you know what

[20:00] kind of activity I have been doing along my month, you know? Or maybe I have to focus more on my cardio, drinking more water so that I can take more informed decision if I am looking for a weight loss, if I want to have a better health

[20:16] care system. Right? So, we can analyze the data, our own data, how we are living, what kind of water, how much water are we drinking, how much steps we are walking. Based on that, certain decisions can be

[20:30] Google also employs data science to offer relevant search recommendations as users as they type in the query. So, it gives the real time, the moment you're typing it, you know, predicts the next word, you know, that what do you want to

[20:46] search? The data science in health or in health care course or a health care health care course or a health care research paper, etc. finance domain using data science can help a loan manager can easily access

[21:00] you know, whether you are you know, applicable for a loan or not. So, once you have applied for the loan in the portal like what all is important like what is your salary how many dependents you have do you have a medical insurance

[21:15] or not what are the different properties based on those kinds of analysis what kind of you know your civil uh report is credit report approved amount report is credit report approved amount risk based on those criteria decisions

[21:30] can be taken whether you will be approved for loan uh loan or not. Clear learners are we understanding the applications of data science? Have we understood what data science is?

[21:43] understood what data science is? So the data science process what what all is required? First thing as I told you is nation. Problem definition for different projects for different

[21:56] domains for different data. What are we looking at? What are we looking at the loan application part of the uh person or what are we looking at the health care part of the person? So the problem needs to be defined.

[22:09] Based on the problem data authentic data needs to be collected from the correct resources right? If you want to know how many people have uh got you know negative side effects after the COVID vaccine we

[22:22] have to collect that reliable authentic data. Once the data is collected data cleaning and exploration is required. Now what are the issues in the data? Can anybody tell me? Since you've done SQL what are

[22:38] the problems uh you know that a data can face? Can anybody tell me? Come on tell me. Technically missing data absolutely correct Sahana inconsistencies in data very good Mira the null values in data

[22:54] right? Errors in data duplicate value data absolutely correct. So all that needs to be cleaned and explored before we send it for further

[23:06] analysis. What will happen if you are not cleaning this data? Dupli- Sahana, if you're not removing the duplicates or null values, what will happen, Mira? Sahana.

[23:19] What will happen? Normalization is a little different. If you're not cleaning the data, what will happen, Darshan? very, very important step in data science. And once done, we then move on

[23:35] to the feature engineering part of it. What does What does feature engineering That uh if any kind of, you know, especially categorical data needs to be converted into numerical data, scaling needs to be

[23:49] done, binning needs to be done, removing of the outliers. So, this is still here we are going to complete our data science journey, right? This course, you know, deals till this particular part of the whole data science process, right?

[24:06] In the next step, as you will move forward in machine learning, model building and training will be covered or the deep learning models training will be covered. Finally, we can move ahead with model evaluation and uh model

[24:20] deployment. Getting my point? So, the first step in the data science process is clearly define the goal or the question to be addressed through the data analysis. Gather the relevant data set information

[24:34] from different sources, pre-process, handling missing values, outliers. Then we need to uh transform the new features to enhance the data set's uh information, model building and training, and finally model evaluation

[24:49] training, and finally model evaluation and deployment also needs to be done. Clear? And here in this particular course, we are going to do understand data science from the Python perspective. And can

[25:05] from the Python perspective. And can anybody tell me why is Python a hit language? Anybody who can tell me why Python is a hit language in today's world? Come on. We've already done that.

[25:20] Come on. We've already done that. Why is Python a hit language in today's Is it easy to understand? It's a high-level language. It's interpreted language. Huge collection and libraries for data science and machine learning.

[25:34] Very good. Very good, Abhijita. Very good, Sahana. What about others? good, Sahana. What about others? Functions can be treated as objects. Right? They can be passed as parameters to the functions, also.

[25:48] are answering. Most of you, rather I would say, are not answering. So, try to understand the advantages of Python for data science. That it is an open-source.

[26:00] It's an interpreted high-level language that supports object-oriented programming. Ease of use and simple syntax. Scalability when compared to R. Availability of the wide variety of data science libraries and packages.

[26:14] Compatibility with all major operating systems. Creation of new data science libraries daily by vast number of online user communities and powerful visualization libraries are there. Right?

[26:30] And the different Python packages for data science that we would be covering in this particular courses first, we will concentrate on the NumPy library, which is used for scientific computing. That supports large multi-dimensional

[26:45] arrays, matrices, and includes comprehensive mathematical library. Second is we are going to cover Pandas, which is efficient storage and manipulation of structured data such as the time series and tables.

[27:03] SciPy. SciPy is an open-source library built on top of NumPy and is used for scientific formula. That also we would be covering up in this particular course. Statsmodels also we are going to cover is a Python module that provides

[27:20] classes and functions for estimating many different statistical models, many different statistical models, conducting statistical exploration. And scikit-learn. Scikit-learn we will cover a little bit part of it, not much

[27:35] because it's part mostly used for machine learning. So scikit is widely used for open-source machine learning library for Python. Known for its simplicity, ease of use, versatility in handling various machine

[27:48] learning tasks. It identifies objects in images for autonomous vehicles and facial recognition systems. It detects fraudulent transactions in banking and e-commerce e-commerce platform. It analyzes customer reviews for sentiment

[28:04] classification in marketing and social media analysis. Clear, learners? Are you there with me? And if I talk about Matplotlib, Matplotlib is a library. It is a

[28:19] comprehensive tool for building statistic static static animated and interactive visualization. We would be covering up Matplotlib in detail as we

[28:31] move along this journey to build different scatter plots, bar charts, histograms, pie charts. Advanced library over Matplotlib, Seaborn is another data visualization library in Python that is built on top

[28:44] of Matplotlib. It provides a high-level interface for creative, attractive, and informative statistical graphics like histograms, box plot, violin plots, etc.

[28:57] Then we would be covering up Plotly, which is used for creating interactive publication quality graphs and visualization. It is suitable for web applications also. Clear?

[29:12] So now let's do a quick recap of the different plots that you might know before. For example, let's first understand the For example, let's first understand the line plot. Yes, learners. What is the

[29:25] use of line plots? I'm sure you must have all done it in your school days, in your mathematics, in your graphs. You know, maybe science, physics, or math statistics. So can anybody tell

[29:39] or math statistics. So can anybody tell me what is the use of line plot? It show relationship over two quantities, comparing growth. So this is like x-axis, this is my y-axis. What kind of growth? Good phase.

[29:54] y-axis. What kind of growth? Good phase. What kind of growth for x and y-axis? Especially if you are dealing with stock markets or anything which has been over a period of time, the sales of car over a period of time, or the sales for any

[30:09] monthly sales of fruits or vegetables in terms of economy. Very good, phase. Very terms of economy. Very good, phase. Very good. Excellent. Anything else? So a line plot displays data points connected by straight lines, often used

[30:23] to visualize trends or relationship between two variables over time and between two variables over time and other continuous intervals. Agreed, learners? Then a little you know, appearance,

[30:37] making the line plots better using the marker plots. What is this marker plot? It displays the data points with markers useful for scatter plots for visualizing individual data. If I really want to find out the relationship at this

[30:53] particular point, do we Do we also try to draw the graphs like of your marks? So, Faraz has changed his name to Trump. That's interesting.

[31:07] That's interesting, faculties. Your students are changing changing their his uh uh you know, name to Trump. Okay.

[31:22] marker plot is something which can, you know, used to highlight certain points, you know, maybe even it could be the marks or or a particular specific uh you know, condition in in the graph. Then, scatter plots. Where do you see

[31:38] the scatter plots? Which gives the relationship between two Which gives the relationship between two numerical values. Come on, learners.

[31:50] the weight. People who are uh more with the weightage have more BP issues or, you know, uh two numerical values can be used to compare using the scatter plots.

[32:03] Right? Height and weight in a particular population. Then, have you seen some graphs like this? Such as area plots. Which help us to give the cumulative data. If this is the sales of the

[32:17] company, maybe the blue one is giving the sales for the first quarter. This is for the second quarter. This is for the third quarter.

[32:29] Getting my point, learners? NMIT, are you there? So, an area plot visualizes the cumulative data changes over time such

[32:41] cumulative data changes over time such as tracking the total sales where we try to or maybe you know, if I want to analyze or give you a report

[32:53] card, so what was your result semester wise? What was your result in your first semester, second semester, third semester? Cumulative results can be semester? Cumulative results can be easily represented using the area plots.

[33:05] Bar plots, do we see bar plots? I think so, that's very, very common that we see around us. Do we see bar plots, learners? Nothing but rectangular graph that show vertical and horizontal data based on

[33:21] vertical and horizontal data based on another axis. sales of different products over the month. it as, you know, students as display student

[33:35] you know, students as display student grades in different subjects. using in your science or statistics classes to draw to see the downfall or

[33:47] rise and other things to mark certain particular points. Yeah? So, grid plots basically help in comparison of multiple plots by comparison of multiple plots by enhancing the visualization points.

[34:03] Another very, very important graph is a histogram. Anybody who's aware about Yeah, last but not the least, one of the favorite graphs which we all have dealt with is a pie chart where the whole data

[34:16] is considered as 100% and a part of it a fraction is, you know, shown in Clear? Right, it could be the, you know, the share of languages in a in the whole 100% area or the,

[34:32] you know the different expenditure of the house in different expenditure of the house in different areas that can be taken. So, pie plots show the proportions of the whole like the market shares or the survey

[34:44] responses, right? Nothing to worry. We will be dealing up with all these charts in detail when we take up data visualization. This is the you know the NumPy 2.4.0 release. It's a

[34:59] powerful It helps us to create N-dimensional array computing tools open source library which helps us. This is the you know the material where you can you know practically use it. It helps us in

[35:14] you know working in scientific domains. Array libraries in data science, it helps to extract uh transform and load the data, exploratory analysis, model and evaluate, report and

[35:28] dashboard. All these things can be done with the help of machine learning, visualization, scientific domains, array libraries, etc. Clear?

[35:41] Lot of lot of practical use, all the TensorFlow, all the deep learning activities, unstructured data is all stored in terms of N-dimensional Clear? So, what is the use of NumPy? NumPy is

[35:58] nothing but a numeric Python. It is a package used for computation on homogeneous and N-dimensional arrays. What are the properties of array? They are mutable. What does the term mutable

[36:14] mean? Yes, learners. What does the term mutable mean? Yes, we can and which is the other data structure which are mutable? Other data structures in Python which

[36:26] are mutable? List, dictionary, and which is immutable? immutable? Tuples are immutable, strings are immutable, int, float, all are immutable, right? Very good. Good.

[36:43] immutable, right? Very good. Good. What does the term homogeneous mean? When I say the term homogeneous, yeah, they all are of same data type.

[36:59] Secondly, it can be accessed using integer position. Right? Integer position. Strings are accessed using integer position, lists are accessed. That is, in other words, indexing is allowed. What are the two

[37:15] indexing is allowed. What are the two types of indexing available in Python? What are the two types of indexing available in Python? Very good. Positive and negative. Arrays deal with numeric data and high

[37:31] Arrays deal with numeric data and high performance in calculation, right? So, what is the advantage of arrays over the what is the advantage of arrays over the list? Right? We just now saw that in

[37:44] list, basically, we store pointer to the data locations. Every data is stored in a different location, therefore, accessing the elements of the list are different. And when I talk about arrays, they are

[38:00] stored in contiguous memory location. First of all, they are of homogeneous data type. Int will store some bytes of information. Float will store some bytes of information. So, they all are stored in

[38:15] information. So, they all are stored in contiguous memory location. That is why contiguous memory location. That is why arrays are faster than lists. That's why this data structure was created because it is much more faster than the list.

[38:31] Clear? Is this point getting clear to Is this point getting clear to everybody? then you will start writing the code. I want everybody to open the Jupiter labs

[38:47] want everybody to open the Jupiter labs and start writing this particular code. I will ask anybody to show the file. I can randomly ask anybody to share your can randomly ask anybody to share your screen. I want everybody to start typing

[39:01] and writing this code and let's understand this code side by side. Learners, please start typing this code on the lab Jupiter lab that you have. The first and the foremost thing that is

[39:16] The first and the foremost thing that is required for this particular required for this particular courses import the NumPy as NP that we import the library. The first step whenever we need to

[39:29] create uh you know arrays is because it's not part of the basic Python. We it's not part of the basic Python. We will always need to import NumPy as NP. I want everybody to type in the code. I want faculties, please look at the

[39:45] uh learners, students are they doing it in the lab or not this particular assignment or not. And this is how we create the array. np.array

[39:58] is the name of the function and these are the elements to be created in the are the elements to be created in the array that is 0 1 2 3. If I'm going fast, you can always tell me to slow down or wait, but I want

[40:11] everybody to type in the code. Ashish, are you doing it? Aditya, Nagendra, Harshita, Suhana. Are you all doing it? Manohar, Chetan, Are you all doing it? Manohar, Chetan, Chaitra, Keertana, Bhumika.

[40:26] Good Manvaya. Good Manya. Good Ashish. Yeah. So, when I print A, this is the Yeah. So, when I print A, this is the output I have done get. And when I do a type A, it seems like a list, but it is not a list because it's an object of

[40:40] not a list because it's an object of class in numpy.ndarray. So, by doing type A, I get the output as type numpy.ndarray.

[41:00] these are nothing but the attributes of the array. A.ndim gives me the dimension of the array. This is a one-dimensional array. The shape.

[41:12] The number of elements or the number of, you know, rows. Basically, it's a column vector. It is written like this. It's gets stored like this. 0 1 2 3. So, the number of rows over here is four. And length gives me the number of elements

[41:29] in the array. Everybody got this? First, we need to import the library whenever we have to create arrays. Once

[41:43] will load it. Not every time you write Then, by using the np.array function, you are able to create arrays. Print the array. Give the type A, that is class

[41:59] numpy.ndarray. A.ndim gives me the dimension of the array. A.shape gives me the number of elements. And len.A gives me the length

[42:12] array. Clear? Shape always gives me the rows and columns. Right? And as I told you, the dimension, shape, and axis are very closely related. One-dimensional array has axis

[42:29] parameter, that's the number of elements. elements. What is one-dimensional array known as? What is one-dimensional array known as? What is one-dimensional array known as?

[42:41] Very good. That's known as a vector. Only Vaibhav understands in this session? Only Vaibhav understands? He only He is only going to respond, nobody else? Then we had

[42:56] two-dimensional array, right? Which consists of rows and columns, that is axis zero refers to the number of rows, axis one refers to the columns. So, what is the other name of arrays, two-dimensional array?

[43:09] That's also known as the matrix. Very good, Meena. Very good. And if I talk about N-dimensional or three-dimensional array, three-dimensional array will consist of axis zero, one, and two. Axis

[43:25] zero, you can consider it as the slices of the bread, axis one, the number of of the bread, axis one, the number of rows and column in that particular bread Clear? So, what are the different attributes of an array?

[43:40] .shape gives me the axis or the number of rows and column, and .ndim gives me the dimension, .dtype gives me the data type, 32 refers

[43:52] the number of bits required to store number of bits, item size gives me the number of item size required to store one element. So, we do divided by eight because one byte is equal to eight bits, and .size gives me again the number of

[44:08] Clear? Is this point getting clear to Is this point getting clear to everybody?

[44:21] So, now today we will till now we were only revising what we had done yesterday. And now we begin with today's session. We have to understand functions. We'll continue. I request everybody to open the 3.02 file. Please

[44:37] go to your LMS, download the material, and be ready with the material in your Jupiter labs. 3.02, 03, 04, 05. We are supposed to complete

[44:50] it today. I request everybody to down go to your I request everybody to down go to your LMS, download the material, and be ready LMS, download the material, and be ready with the files in your Jupiter labs.

[45:03] Anybody who's facing any difficulty regarding the material or the lab, please let me know. Anybody who's facing difficulty regarding the material or lab, please let me know.

[45:20] ready to begin with today's learning? A quick thumbs up from everybody. up? Are you all there in the uh you know, session or not? Or are you

[45:33] sleeping? Parnika, Suhas, Mallikarjun, Parnika, Suhas, Mallikarjun, Rakesha, Rishita, are you there or not? Rakesha, Rishita, are you there or not? Rakshita, Parvi- Parvikshika, Pranav,

[45:46] Lokesh, Ranjan, are you all there or not? Now, let's understand with few function. First is transpose function. So, First is transpose function. So, transpose function as we understand

[45:59] from the perspective of matrices that it interchanges the rows and column. interchanges the rows and column. Flatten will convert any array any shape of array into one dimensional array. It will convert it into one dimension and

[46:14] will convert it into one dimension and reshape will change it into rows and column. Clear? So, the NumPy transpose is nothing but if this particular array is of the shape

[46:29] 2 comma 3. Why 2 comma 3? It consists of two rows and three column. And when I run this np.transpose function, it becomes into 3 comma 2.

[46:46] now the arrays of any shape. It's not that two dimensional arrays. It's applicable to two dimensional. Transpose is also not applicable to two only to two dimensional, but any n dimensional array.

[47:01] array. Similarly, when I use the word flatten, any n dimensional array can be changed as always changed into one dimensional array. Please be clear. So, what what can be the two ways in

[47:15] which flattening can happen? One is row major that I take this first row. Right? So, the elements of the first row

[47:27] are placed first and then the elements of the next row and so on. But if it is column major, I have to change the order equal to F. So, first the elements of my column get flattened and then the elements of the

[47:44] second column get flattened. And when I talk about reshape, you can change it to any shape. A one dimensional array can be reshaped to 2 comma 3, 3 comma 2, and even in more dimensions

[47:59] 3 comma 2, and even in more dimensions 6, 2, 2, 6. But can can I change this 3, 4 to 4, 4? Can I reshape this 3, 4 to 4, 4?

[48:11] Can I reshape this 3, 4 to 4, 4? Why? Because 4 into 4 is nothing but your elements are 16, which it will give me an error. Is this point getting clear to everybody? Now, there is this activity.

[48:25] I want everybody to open their Jupiter labs and start writing this first code. I want everybody to open their Jupiter labs. Faculties, please take a note of labs. Faculties, please take a note of this that learners are practicing this

[48:40] particular code. I want everybody to first do this code. Copy code. Quickly. So, the first code says import the NumPy. We are creating an array A of one dimension. X, we are creating a copy.

[48:56] And here we are trying to change the value of one of the positions of the second array, like 0, 1, and 2. So, I want to change this value of 3 to 99. So, when I do it for A, it does get

[49:12] changed, but when I do it print X, the value does not get changed. Can anybody tell me the reason why? Value in X not getting changed?

[49:24] Value in X not getting changed? Because X over here, we are creating a Because X over here, we are creating a new variable in memory. That means a new memory location is getting allocated.

[49:40] The The memory location is not same as that of the A. It is different. Good. Good, Nana. Good. Very good. So, did you see the change? Did you see the

[49:52] Yeah. Similarly, but when I create a view of the array, now run this code, make Quickly, make the changes in this. Now, instead of copy, create a view, and now you will see that both the values

[50:09] are getting changed. It will give me an output change in B also, as well as X also. That import numpy as np. Array np.array,

[50:23] and the dimension is one-dimensional, and it consists of 12 elements. So, can I change these 12 elements into 12 {comma} 1? Can I reshape this? Are you there with me with the code?

[50:38] So, now we see this one-dimensional array now changes to two-dimensional array now changes to two-dimensional with the shape 12 {comma} 1. We can also reshape to 3 {comma} 4. Because the total number of elements, 3

[50:54] Because the total number of elements, 3 into 4 is 12, 2 into 6 is 12, so we can reshape it. Can I reshape this particular array into three dimension? 3 {comma} 2 {comma} 2, is it allowed? Yes, learners. Is this allowed?

[51:11] Yes. Can we change it from Can we change arrays from lower dimension to higher dimension and from higher dimension lower dimension using reshape? Yes or no, tell me. Reshape function can be used to change

[51:25] any dimensions to any? Yes or no? Of course, when the number of elements are the same. Can we Can I change the dimension from 1 to uh 3 and then 3 to 1? Is it possible?

[51:39] uh 3 and then 3 to 1? Is it possible? Vaibhav, Bhumika, Meera, Prajwal, reshape function. Only Vaibhav

[51:51] understands in this particular batch, nobody else? nobody else? Manya, what about you? Mamata? Monica, Nagendra? Why are you all not responding? Aishwarya?

[52:05] Everybody is getting this code or are you all facing difficulty? Please let me you all facing difficulty? Please let me know.

[52:17] Can I change it to further dimension such as 1 2 3 4 5? Can I change a three-dimensional array or a one-dimensional array to possible? Yes, because 2 into 3 is 6, 6 into 2 is

[52:32] Yes, because 2 into 3 is 6, 6 into 2 is 12, 12 into 1 is 12, and 12 into 1 is again 12. So, the total number of elements remain 12. That is why we get elements remain 12. That is why we get these whole of arrays.

[52:45] All right. And we see that a reshape function minus one will automatically bring it back to one-dimensional array. So, if you pass the parameter in reshape

[52:57] So, if you pass the parameter in reshape as minus one, you will flatten it out to get it as one-dimensional array. But there is one more function which exists there is one more function which exists in NumPy array that is flatten. It

[53:11] consists of these attributes. C means flatten array elements into row major C flatten array elements into row major C style. F means in column column major style as I told you. So, it's either, you know, row major style or a column

[53:26] you know, row major style or a column major Fortran style. A and K are not used nowadays so much because A means to flatten array in a column major order. flatten array in a column major order. If A Fortran contiguous in memory or row

[53:39] major wise. So, now we have, uh you know evolved from these uh uh issues. So, you know, the two main uh orders are the uh you know, types of uh

[53:51] arrays that exist are C and F. Then, we have K, which means to flatten array elements in order of the elements laid out in the memory. out in the memory. Clear?

[54:10] Is this point getting clear to everybody? tell me the dimensions of A? Quickly tell me. Who's going to tell me? What is the dimension of this array A? Highlighted array A.

[54:25] Yes, learners. What is the dimension of It's a two-dimensional array. And when I do a flatten, right? Normally, by default, it will do it row-wise. So, it will take order is

[54:40] equal to C. Okay? is equal to F. So, what is the difference? If this is my

[54:53] two-dimensional matrix, so if I do row major, it becomes 1 2 3 4, and column major, it becomes 1 2 3 4, and column major, 1 3 and 2 4. Do you see the major, 1 3 and 2 4. Do you see the difference?

[55:14] Another important point that you have to understand over here is that B is the understand over here is that B is the copy, not the view of array A. What is copy, not the view of array A. What is the difference between copy and view?

[55:30] What is the difference between copy and view? Yes, learners. the original array. It creates a new one, whereas view does. So, if I make

[55:44] changes in my B, will it be affected on A? If I make elements in B, will it be affected in the original array?

[56:01] Yeah, it will not have that effect. Good, Darshan. Ishan, are you there? copy of the array flattened into one-dimensional array. So, that is the

[56:15] difference between reshape and flatten function. Are you getting this point? Flatten will always create a new array. It will return a copy of the array flattened into one-dimensional. So, if we have this three-dimensional array

[56:30] we have this three-dimensional array and if we flatten it using order F, what is order F? Column-wise, that is, I will take this Column-wise, that is, I will take this first element, one and seven.

[56:42] first element, one and seven. Then four and then 10. difference? In three dimension, then five, 11, three, nine, six, 12. Clear?

[57:04] I don't pass this parameter, will I get the same answer? Yes, I will get the same answer because by default it takes row-major-wise.

[57:17] by default it takes row-major-wise. Clear? Do it practically in your Jupiter Lab notebooks.

[57:33] Everybody is able to do it? Everybody is able to do it or anybody who's facing difficulty? Malatesh, Monica Monica Come on, answer. S. Lokesh, Sachin

[57:45] Rakshita Vinal, Bhavishya Chandra, Varsha Mallikarjun Mallika, there's no response from you. Darshan, Dhanush Come on, respond learners. Anurag,

[58:00] Come on, respond learners. Anurag, Anusha run the code? Sorry. Are you all able to run the code

[58:12] Sorry. Are you all able to run the code or not? able to run. One I think so Ganga is doing it. Okay, Darshan is also doing, that's good. Bhumika's also doing.

[58:27] Only four? Malatesh is also doing. That's it. Nobody else is able to run the code?

[58:53] Still, I think so lot of learners are still not doing it. Faculties, please take note of this. So, nobody is responding. Nobody is responding. So, if this is variable of two

[59:08] dimensional arrays, so transpose will mean that my two into three matrix mean that my two into three matrix changes into three into two matrix. Clear? Right? And now, if I reshape this 12

[59:23] Right? And now, if I reshape this 12 elements into three dimensional as 4 3 1 elements into three dimensional as 4 3 1 can I transpose So now learners, you have assignments for today.

[59:36] I want please make note of this faculties that they have to complete these assignments before the next session because I'll be taking it up in the next session. So there is this assisted practice. Do you all see this?

[59:51] The problem statement where it says as a data scientist, your task is to create a Python project that explores the NumPy arrays attributes and function. So this is the list given to you. Explore the key attributes, find out the end

[1:00:06] dimension shape size and demonstrate the functions. So what is the idea? That if suppose you have a list and I wanted to multiply the elements, so I had to explicitly use the for loop to multiply two with each of

[1:00:23] Right? To get the answer. But when I am talking about arrays, first of all, the two arrays have to be of the same shape and size. That means

[1:00:36] broadcasting will happen. What is broadcasting? That it will make the two arrays of the same shape. Both will become vectors with two and two. And by become vectors with two and two. And by default, the implicit for loop runs to

[1:00:50] default, the implicit for loop runs to give you the output. Is the concept of broadcasting and vectorization getting clear to everybody? Neha, Vaibhav, Bhoomika, Meera,

[1:01:03] Neha, Vaibhav, Bhoomika, Meera, Aishwarya. To understand the concept better, let's understand this with an example.

[1:01:17] Learners, please look here to understand this with an example. Let's suppose these are my arrays. Now, can you tell me the shape of this array? This two-dimensional array, quickly learn us.

[1:01:32] What is the shape of this two-dimensional array? And what is the shape of the other one? That is also 4,3.

[1:01:47] Are the two shapes same? Yes, they are. So, can uh arithmetic operation happens? Yes, element by element. This element will get added with this. This element will get added with this. This element will

[1:02:02] get added with this to give you the output. Clear, learners? Are you all getting this point?

[1:02:21] Right? So, that means in these two arrays, no broadcasting is required. arrays, no broadcasting is required. Only vectorization happens, and the Only vectorization happens, and the arithmetic calculation is performed.

[1:02:33] The first case is clear? Now, look at the second case. What is the shape of this array? This is 4,3. What is the shape of this array? Ganga, Nita,

[1:02:46] Ganga, Nita, Vibhav. columns? Three columns. Very good, Vibhav.

[1:02:58] Three columns. Very good, Vibhav. Now, are my two columns matching? Now, are my two columns matching? Are the two columns same? Yes, they are. Uh broadcast this one to

[1:03:10] the same number of rows now. So, broadcasting is possible in this So, broadcasting is possible in this case. Therefore, it expands this array case. Therefore, it expands this array into four rows to perform the operation.

[1:03:24] Getting my point? Is this point getting clear to Is this point getting clear to everybody?

[1:03:43] happen. Good, Swyam. And of course, vectorization will happen for element-by-element calculation. Now, look at the third example. Now, look at the third example. What is the shape of the first array?

[1:04:06] column, and this is again one row and three columns. Good, Chaitra. Good, Bhumika. Now, how is broadcasting possible? You have to check that if one of the you know, rows is

[1:04:19] four and other one is one, right? So, can we expand it to four rows? Yes, that's possible. And if this column is three and this is one, can I expand it to three columns? Yes. So, broadcasting

[1:04:33] can happen. It cannot It It is not necessary that the broadcasting happens every time. And it is not necessary that it will happen only for one array. It can also happen for both the arrays also. So, here the both arrays again

[1:04:47] become of the same shape, then the vectorization happens, and then you get the output. Clear?

[1:05:00] Is this point getting clear to everybody?

[1:05:21] Any questions? Any doubt there, yeah? Very, very important concept. One more Very, very important concept. One more example. Please look here. run over here. Yeah. Now, np.arange.

[1:05:36] It is one row and three columns. Agreed? And a range function is giving the value 0 1 2. And when I do a plus five, what is

[1:05:48] 2. And when I do a plus five, what is plus five? It is a scalar value or a zero-dimensional array. Broadcasting will happen? Yes. It will broadcast it will happen? Yes. It will broadcast it into 5 5 5 and then the vectorization

[1:06:02] operation happens. Clear? I'll again repeat it. Here in my first example, this is my one-dimensional array with the shape of 1 comma

[1:06:16] three values. This is my scalar value, so broadcasting will make both the shapes same as 1 comma 3 and then vectorization will Uh output.

[1:06:31] output. Clear? Is this point getting clear to everybody?

[1:06:44] I head to np.ones 3 comma 3. That's one of the other ways to create uh an array with all values one. The So, the shape of this array is 3 comma 3. And this is np.arange

[1:06:58] 3, 1 comma 3. Now, there should not be any confusion. Since the columns match and one of the dimension is one. Can it be expanded? dimension is one. Can it be expanded? Yes. And therefore, broadcasting happens

[1:07:12] and then the vectorization operation. Clear, learners? Chaitra, Monica, Vibhav Mira Nagendra Are you all getting this point?

[1:07:28] And again, in the third case, can both the arrays be broadcasted? Yes, when I have this shape of 3 comma 1 and this is 1 comma 3. Now, how come broadcasting is happening? Since one of the dimension is one, we can broadcast

[1:07:44] the rows. The column is also one, we can broadcast it into three. So, both the arrays become of the same shape and therefore, the vectorization logic happens. Swayam, are you getting this?

[1:08:06] Is this point getting clear to everybody?

[1:08:20] Now, suppose if this is the array of the shape 4 comma 3 and this is 1 comma 4. So, now the columns are not matching. This it into four, but since the columns are

[1:08:35] it into four, but since the columns are not matching, it will give me an error. Clear? Let's do it practically, learners. Let's do it practically, learners. Are you ready with the file 3.02?

[1:08:48] Sorry, 3.03. Learners, are you ready with 3.03? Vibhav, Monica, Neha Prajwal, Mira

[1:09:09] understand now this particular code? Yesterday we had worked on this.

[1:09:21] asterisk two, that means this particular value values are repeated. They don't value values are repeated. They don't Sorry, they don't get multiplied.

[1:09:41] of the way is by using I create an empty list. I run the for loop and then each of the elements does it. But a beautiful way was solved by way above that we can also use list comprehension to get the output.

[1:09:55] Is this point getting clear to everybody? Are we clear with this these points? Yes, learners.

[1:10:17] beauty of arrays? Another beauty of arrays that by simply Another beauty of arrays that by simply creating this array 2 3 4 and just by doing a multiplication two, I get the values multiplied by two. So, what is

[1:10:31] the concept that is working behind this? Yes, learners. Tell me. Yes, learners. Tell me. What is the concept

[1:10:49] What are the two main concepts working behind arithmetic operations? First is broadcasting. Yes, that is 2 into 2 into 2 and then vectorization that is element by element.

[1:11:03] Clear to everybody? Vectorization is now clear?

[1:11:15] All right. So, different arithmetic operations are possible. One way is by using this plus operator or you can also use the np.add function for addition. Minus is for subtraction. Subtract. np.negative is a unitary method. You get

[1:11:32] all the values as negative. Then we have multiply that is multiplication. multiply that is multiplication. np.divide that is single division. Can anybody tell me what is the difference between divide and floor

[1:11:46] divide? Who's going to tell me this? divide and floor divide. Yes, learners, tell me.

[1:12:14] the difference between divide and floor divide. That's basic Python I'm asking. Vibhav gives large gives integer value. Are you sure? Floor division gives only Divide gives the division value, floor divide gives the remainder. No.

[1:12:30] Swayam is correct. It is the % operator, the modular It is the % operator, the modular operator, which gives me the remainder.

[1:12:43] you know, the decimal values. This gives me the integer part of the quotient. Getting my point? Is this point getting clear to everyone?

[1:13:05] please be clear. And double asterisk is used for finding A to the power B.

[1:13:17] Is this point getting clear to everybody? one-dimensional array or a vector, B is my scalar value, so what will happen?

[1:13:29] np.add Is vectorization and broadcasting happening over here? Is broadcasting happening in case of Is broadcasting happening in case of this particular code?

[1:13:45] Tell me. In A and B, vector is broadcasting happening, no, Mira? Vaibhav says yes, what about others? One yes, one no, what about the others? Why no, Mira? Why Vaibhav yes?

[1:14:00] The 10 needs to be broadcasted to the same shape. Right? Then only the vectorization will happen. You got my point, Mira?

[1:14:19] Yes, so broadcasting is happening over here. So either you can use an np.add here. So either you can use an np.add function or even a plus sign can work. function or even a plus sign can work. Clear?

[1:14:37] Now tell me what is A and what is B. What is the shape of A and B? Tell me quickly. What is the shape of A and B.

[1:14:51] not 2 {comma} 2, it's 2 {comma} 3, right, we have up. So, is broadcasting happening over here? We are using subtraction. Is bro- We are using subtraction. Is bro- broadcasting happening over here?

[1:15:13] No. Why, Meera, no? Why, Neha, no?

[1:15:33] Very good, Neha. Very good, Meera. Very good, Meera. So, but vectorization is happening. So, but vectorization is happening. Is vectorization happening?

[1:15:56] Yes. Yes. Vectorization will always happen. What is vectorization? Implicit running of the for loop. Implicit running of the for loop. Getting my point?

[1:16:14] are of one-dimensional array. So, either we can use np.multiply A into B. So, is we can use np.multiply A into B. So, is broadcasting happening over here?

[1:16:28] In multiplication problem, is broadcasting happening over here?

[1:16:51] >> Is broadcasting happening in the in multiplication problem or not? Bhumika, no, it is not because again, they both are of the same shape.

[1:17:07] they both are of the same shape. Yes or no? Are you understanding it or not? And vectorization will definitely happen,

[1:17:20] that element-by-element multiplication will happen. will happen. Clear?

[1:17:35] Is this point getting clear to everybody? Neha, yes or no, Vaibhav? everybody? Neha, yes or no, Vaibhav? Rajwal, Nagendra.

[1:17:51] A is two-dimensional, B is one-dimensional, is broadcasting happening over here? In division, is broadcasting happening? the same shape. And that's how we get the output.

[1:18:12] And if I talk about power of A to the power B, so 2 to the power 2 is 4. 2 to the power 3 is 8, 2 to the power 4 is 16, 2 to the 8, 2 to the power 4 is 16, 2 to the power 5 is 34, 2 to the power 6 is this

[1:18:27] much. Clear?

[1:18:40] with 1 * 3? See, it's not matrix multiplication See, it's not matrix multiplication element operation. So, if this is ABCD,

[1:18:58] G and H. It's not matrix multiplication. A will get multiplied by C, B will be getting multiplied by F, C will be getting multiplied by G, and D will get

[1:19:10] multiplied by H. It's not that matrix multiplication like this. It's element by element operation. That's why it is vectorization. Now, does array support that? Yes, array support both positive as well as

[1:19:26] support both positive as well as negative indexing even at the same time. That was not possible in list or tuple, right? But in arrays, it is possible to use positive and negative indexing at the same time.

[1:19:50] values, positive indexing always starts from zero on the left-hand side, we move from zero on the left-hand side, we move to the end. So, when I say A to the power two, what will be the value? What is the value at index number two?

[1:20:11] What is the value at index number two? Come on, learners, tell me.

[1:20:24] change the value to minus two, what is the value at minus two?

[1:20:36] possible. Similarly, if I have a matrix or a two-dimensional array, this is nothing but my row. This is nothing but my column. So, this is zero, this is one, this is two, and this is again zero, this is one, and this is two.

[1:20:51] Right? So, how do I take the position? This is my first row and second column. This is my first row and second column. This is my zeroth row and zeroth column. Can I do a mixture of positive and negative indexing? Yes, that's also

[1:21:06] possible possible that this is my second row. Row will always come before, please. We cannot interchange that. Row will always any uh term written before the comma will always represent the row.

[1:21:21] And any term written after the comma will represent the column. So, and this is how we get this element as Clear? Is this point getting

[1:21:35] Is this point getting clear to everybody? Same? Simple? Right? So, two-dimensional array, we have two axes. The first axis always represents the row, which is also known as axis equal to zero.

[1:21:51] as axis equal to zero. Right? And this represents as axis equal Right? And this represents as axis equal to one, that is the columns. Okay. Now, let's see who's able to answer this question. Right? Let's see

[1:22:05] who's attentive in the session now. So, if this is my indexing and slicing, before the comma, I'm talking about the zeroth row. Agreed, learners? So, this

[1:22:18] is zeroth row. Like this. And which columns am I talking about? What does three colon five mean? Slicing is also same as the basic. What

[1:22:31] does three colon five mean? What does the slicing mean? We are talking about the slicing mean? We are talking about the Very good, Mira.

[1:22:46] So, in this row, we get the elements as three, four. Very good. So, are you Are you clear with this answer? Now, tell me which row are we talking about? Tell me the row that we are talking

[1:23:00] Tell me the row that we are talking about in the next question. fifth row. Agreed, learners? Fourth and fifth row?

[1:23:14] Similarly, we start with fourth. We are talking about the fourth and the fifth column. So, therefore, we get these points. So, therefore, we get these points. Clear?

[1:23:40] Got it, Vaibhav, Neha, Bhumika, Prajwal, Mira?

[1:24:00] answer for this question. Tell me the answer. Which rows are we talking about?

[1:24:22] All the rows. Very good. So, we are talking about and which column? Second, very good. Very good. So, we are talking about these elements. Got it?

[1:24:36] Now, tell me the last one. What is the concept that is getting used? What are these two colons known as? What is the concept of two colons known as?

[1:24:51] slicing. If we have a step parameter also, what is that known as? Good, Mira. It start, stop, and step, but what is it known as? but what is it known as? What is that concept known as?

[1:25:16] No, that concept is known as. Please remember that. That concept is known as remember that. That concept is known as striding. Okay?

[1:25:30] start from the second row, jump two parameters. So, we are talking about the second and the fourth row. What about columns? We start from the

[1:25:42] zeroth column column, yeah. So, and jump two parameters. 0 1 2 3 4 So, and jump two parameters. 0 1 2 3 4 and 5.

[1:26:07] So, this is 12 14 16 24 26 28. Getting this point? Understood the output? Mira, Vaibhav, Neha.

[1:26:35] Again, this is my starting position. I start from the second row. There is no parameter over here. It will go till end, and the jump parameter is two. Here, there is no starting position. It will start from zero, go till the end,

[1:26:50] and the jump parameter is two. Now, clear?

[1:27:02] then we will go in for a break, because now we need to understand how does the three-dimensional array slicing indexing works, right? We know that in three-dimensional array, there are three parameters or three axes which

[1:27:18] are three parameters or three axes which define that. Axis zero selects the matrix, which is the slice of the bread that we want to work on. That selects that we want to work on. That selects the matrix. Second index is J, which

[1:27:31] selects the row. And the third index K, which selects the First point is getting clear? The first parameter over here is not the

[1:27:43] row, but it selects the matrix. J, or the axis one, represents the selects the row, and K is the selects the column, right? the column, right? So, if I have this example, how is it

[1:27:58] represented? It's like NumPy array, three-dimensional array. That's how it is represented, and that's how even a colored image is represented with the channels R, G, and B, right? So, three dimensional array is an example of

[1:28:15] a colored image, okay? So, now let's take this example. If I So, now let's take this example. If I want to access 2 {comma} 0 {comma} 1, what does 2 {comma} 0 {comma} 1 mean? That my value of I is 2.

[1:28:33] J is 0. And the value of K is equal to And the value of K is equal to Yeah?

[1:28:52] right? So, if I look at I, I start from here 0, 1, and 2. So, which matrix am I start talking So, which matrix am I start talking about? I'm talking about this matrix.

[1:29:06] Yeah? Then, it talks about uh rows. So, if I talk about the rows, I'll again start from positive indexing 0, 1, and 2. I am

[1:29:18] talking for this particular matrix, this particular row. particular row. Yeah?

[1:29:30] Is this point getting clear to everybody?

[1:29:46] Then, I talk about K. K is equal to 1. That is 0, 1, and 2. So, first column, That is 0, 1, and 2. So, first column, so the element 31 gets picked up.

[1:30:04] How the three parameters are doing indexing and slicing over here.

[1:30:16] now. This first comma before the comma is representing my I. This is representing my J.

[1:30:30] And this is representing my K. This is I, J, and K. So, this is slicing happening over here. Since there is no starting point, it talks about the first two planes, that is 0 1. The two is not included. Clear?

[1:30:50] When I say about J, it talks about the rows 0 1 and 2. So, it starts from 1 goes till end for this matrix as well as for this matrix. So, whatever matrix uh

[1:31:04] has been selected, that many rows will get selected. And if I talk about get selected. And if I talk about columns, it is 0 1 and 2. So, we start from the zeroth row, first column. Two again is not uh

[1:31:19] included. For both the matrices, 0 and 1. Therefore, we get the output as 13 14 1. Therefore, we get the output as 13 14 16 17 23 24 26 27. 16 17 23 24 26 27. Clear?

[1:31:37] Is this point getting clear to everybody?

[1:31:54] Or do I need to repeat it? Tell me, Vaibhav, Meera, Neha, Bhoomika. Vaibhav, Meera, Neha, Bhoomika. Quickly.

[1:32:14] Chetan Kumar. Aishwarya, Abhijita, Ashish.

[1:32:27] Okay. Tejas says to repeat it. Let me quickly repeat the three-dimensional array for all of you. Let me quickly repeat it. three-dimensional arrays, three-dimensional arrays are, you know,

[1:32:42] three-dimensional arrays are, you know, represented by three axis. Axis zero represents the matrix. J is axis one, which selects the row, and K represents the axis two, which selects the column.

[1:32:58] two, which selects the column. So, if I have these three values, okay? So, if I have these three values, okay? To I is two, J is zero, and K is one. So, I is selecting the matrix. So, we start from zero, one, and

[1:33:13] two. So, over here, since two is given, we are selecting this particular matrix. Got it, Abhijita? Manya? Manya? Tejas, got it?

[1:33:31] Now, J represents the rows. So, again, I do it as zero, one, and two. So, I talk about I'm talking about this zeroeth row. Now clear?

[1:33:54] represents the column, zero, one, and two. So, K is 1, I get it as 30. two. So, K is 1, I get it as 30. Now better, Tejas?

[1:34:06] And if we look at over here, there is this slicing happening for I, J, and K. So, if I talk about slicing, it starts from the zeroth matrix. Two is not So, I'm talking about the first two matrix.

[1:34:23] When I talk about the rows, it is 0, 1, and 2. So, I'm talking about the first two rows. And column again, I am talking about the first two columns. Rows I'm talking about the last two, and columns I'm

[1:34:38] talking about the first two. Therefore, I get the output as this. So, can we add the elements directly? Yes, if I add the element at array position at zero at one position, one

[1:34:53] position is two, and at zero position it is one. So, 1 + 2 will give me the output three. Are we clear with this particular output also? And if we talk about two-dimensional arrays, then we have to pass two axes.

[1:35:08] That is, before the comma, we are always talking about the row, and after that we are talking about the column. So, at the zeroth row zeroth row and second position, we get this output.

[1:35:21] and second position, we get this output. Right, learners?

[1:35:36] around with the code, learners. As I've been telling you, start playing around been telling you, start playing around with the code.

[1:36:01] So, we get this element over here at 1 1. So, at the first row, this is my first row, and this is my first column, so the element is five.

[1:36:13] so the element is five. Clear?

[1:36:26] we can access the elements using 1 0 0. Who's going to tell me the answer if I change it to 1 0 1? Tell me quickly.

[1:36:38] change it to 1 0 1? Tell me quickly. What is the answer for the values 1 0 0 What is the answer for the values 1 0 0 1? Tell me. If this is my array, what is the output with 1 0 0 1? Is it eight? Webov?

[1:37:14] I missed out the square brackets. Yes, the answer is Negative also exists over here. also exists over here. So, when I do negative indexing for the

[1:37:32] answer as four. Even is mixture of positive and negative indexing allowed? In arrays, is mixture of positive and In arrays, is mixture of positive and negative indexing allowed?

[1:37:49] Yes. It does not give me an error. It does not give me an error. All right?

[1:38:09] practice. Everybody has got this assisted practice. Will you So, there is a lot of assignments to be done in the file 3.02 3.03

[1:38:25] So, you have four assisted practices to be done. So, now let's move on to 3.05. Learners, what is the concept of slicing?

[1:38:39] That we have a start and a end position. If the start position is not specified, by default, it will start from zero. If the end position is not specified, it will go till the end. And if there are three parameters,

[1:38:53] start, end, and step, what are they known as? Start, end, and step. What are the three parameters known as?

[1:39:08] Very good. Only Vaibhav understands. Nobody else? That is known as striding. Nobody else? That is known as striding. Clear?

[1:39:40] Yes, learners, are you there with me? So, this is my array. This is 1:7. So, it will start from which position? This is zero. This is one.

[1:39:52] This is zero. This is one. This is two. five. This is six. This is seven and this is eight. So, we

[1:40:04] start from one. And the seventh one is not included. So, And the seventh one is not included. So, the answer is 4 3 5 6 8 9. the answer is 4 3 5 6 8 9. Agreed?

[1:40:21] anybody having any questions, any doubt, please let me know. Is the code matching now, Mira? Is the code matching now?

[1:40:36] Neha, is the file matching now? Okay.

[1:40:51] two, three, three, four, the end is not specified, it will go till the

[1:41:05] end. So, the answer is oops, Java, and cloud. cloud. Clear?

[1:41:24] striding mean? That it will start from the first position, zero, two, three, five, six, and seven.

[1:41:44] It will not include the sixth. It will start from first and jump. So, one, two, three. So, the answer will be seven and four. So, the answer will be seven and four. Output is getting clear, everybody?

[1:42:01] Is the output getting clear to everybody? two-dimensional array, if I talk about two-dimensional

[1:42:16] if I talk about two-dimensional array, row and how which columns? Zero,

[1:42:30] two. So, the third is not included, so only the only the second column. So, the answer is 33. second column. So, the answer is 33. Clear?

[1:42:57] this three-dimensional array, you can print that.

[1:43:18] Please look here. This is my zeroth matrix. This is my first. So, we are talking about this matrix, okay? I is clear. J is one colon. That means we are talking about this particular

[1:43:32] row, one colon, right? And if you're talking about columns, so 0, 1, and 2. So, it will start from 1 and 2. So, the output is 5 and 6. Clear?

[1:43:45] Got it, Abhijita, Vaibhav, Neera, Prajwal, Neha, Bhumika?

[1:43:59] Tell me. Tell me quickly. Are you all able to run it practically? and like 0 to -1, this part is not getting

[1:44:15] like 0 to -1, this part is not getting printed. So, if you look at the our last for this? You all have the assisted practice for this? This is the negative slicing.

[1:44:29] This is the negative slicing. Now, another most important library of Python, that is the Pandas library, without which no, you know, data analysis can be completed. So, if you know, if you say,

[1:44:44] "I know data analysis," it is assumed that you understand this Pandas library. Why is it important? Because it is helps to represent the data in the form of

[1:44:56] tables. When we say tabular data, that means data in the form rows and columns. This is how we generally see the data in terms of SQL, generally see the data in terms of SQL, Excel files, that these are my

[1:45:11] Excel files, that these are my rows, right? And these are my columns. generally see as data, you know, structured data, to put it more precise, that's how. So, So, column

[1:45:25] is generally of the same data type, right? Right? And this can be represented as a one-dimensional array. Right? And the whole of the uh table can be as two-dimensional array

[1:45:41] or in the form of matrix. Can we say that? One-dimensional array, in case of the Pandas library, is known as the series

[1:45:55] function. And two-dimensional array is known as a data frame. known as a data frame. Please try to understand.

[1:46:10] So, the two data structures available in the Pandas library are Pandas series and Pandas data frame. Pandas series is nothing but one-dimensional labeled homogeneous array where the size is immutable.

[1:46:25] And the data frame is two-dimensional labeled array, which is a mutable structure. Now, the question arises that when we already had one-dimensional and

[1:46:37] two-dimensional array, right? Then what was the need of these? These data structures. So, the need of these data structure is in their label

[1:46:52] these data structure is in their label indexing. Please try to understand. Here, they are two types of indexing available. We can define our own label. That means

[1:47:05] here, we don't have to give 0 1 2 3, but every column will have its own name or index. So, the column name is known as the label index. And similarly, I can

[1:47:18] give my own names to the rows also. It is not necessary that it will be 0 1 2 3 etc. Clear?

[1:47:30] Is this point getting clear to everybody?

[1:47:53] Right? So, let's start with the first um data structure of this Pandas library, that is a Pandas series. Series is nothing like a column in a table, and it is a one-dimensional array

[1:48:08] table, and it is a one-dimensional array holding data of any type. Over here, but over here we see it as uh you know, the default indexing, but you can define default indexing, but you can define your own labels also.

[1:48:22] Getting my point? You can define your own labels also. All right. So, a series has a series name,

[1:48:36] So, a series has a series name, a values associated with it, and of course, you can define your own index value. Right? So, how do we go about it? I want everybody to open the Jupyter notebook and start writing this code.

[1:48:51] The first thing that you will write, I want everybody to do this activity. Import Pandas pd. Everybody open your Jupyter lab

[1:49:08] and start working on this activity. The first line is going to be importing of the library, that is import pandas as pd.

[1:49:27] One with the name temperature, other with the uh days. Quickly, quickly, I want everybody let me see paste the codes uh code initially. And

[1:49:39] the function which is used to create the series function is pd.series

[1:49:51] equal to days. Got it?

[1:50:14] is vertically aligned. First difference. And rather than having my index as default 0 1 2 3, this, Tuesday.

[1:50:27] Anybody who's got this output? Quickly. Who's pasting the code first? Vaibhav, Meera, Neha. Vaibhav, Meera, Neha. Suhana.

[1:50:50] I want everybody to work on this activity.

[1:51:08] So have you been able to do it? Series from the list. from the list. Now, I want you to change this code.

[1:51:21] Comma, and write index index is equal to temperature. Naina. And then give me the output. Then check

[1:51:34] the output. Good, Naina. Interchange the attributes of the series function. function. Parameters of

[1:51:57] Excellent. Right. Right. So, do you see that the rows have interchanged? This becomes my index, and Monday comes over here. Right, Naina? This is the output you get. Interchanged output.

[1:52:18] Others have done it? Meena, Vaibhav, Neha. Bhumika.

[1:52:32] NumPy library, we have these two series, S1 and S2. S1 and S2. And when I do S1.S2 to add, we see that the elements are getting added element by element.

[1:52:45] by element. So, it uses the concept of broadcasting and vectorization. Well, not broadcasting, because both of them are of the same shape. So, it is using the concept of vectorization. Now, quickly

[1:52:58] concept of vectorization. Now, quickly run this activity. library built on top of NumPy and is used for data manipulation. It

[1:53:14] introduces data structures like data frame series that make working with frame series that make working with structured data much more efficient.

[1:53:28] gives the data type along with it. Also, it uh type along with it. Also, it uh uh structure, intrinsic data alignment along with your label index. That's the

[1:53:42] advantages of the Pandas library. The two data structures available in the Pandas libraries are series and data frame. Series is one-dimensional labeled array, supports multiple data types, and data frame is two-dimensional array,

[1:53:57] right? Which can be a combination of series also. Right? So, now let's start with the very first Nanas, are you there with me? 3.1

[1:54:10] 3.1 Everybody is there with me? I want a quick confirmation from everybody.

[1:54:27] Great, Abhijita. So, first thing, what is the first step the Pandas library. What is data? Can you tell me the data What is data? Can you tell me the data type of this data? What is this data?

[1:54:46] now pd.capital series data is I pass it as my data, and index I pass it as ABCD, and that's how I can check my output.

[1:55:07] I'll just do a copy paste so that it's easy for us to uh

[1:55:34] output is 1 2 3. Yes, learners. I've just done a copy Yes, learners. I've just done a copy paste over here. Nothing more than that. paste over here. Nothing more than that. You want I can just show it to you.

[1:55:47] You want I can just show it to you. Paste it on the chat also.

[1:56:02] We can also create series using dictionary also. The beauty with dictionary is that you know the keys become my index.

[1:56:15] Uh values become my data.

[1:56:33] Right. Do you see this? Both are giving me the same output. Both are giving me the same output. Learners, are you there with me?

[1:56:45] able to do it, Vaibhav, Nana, Mumika, Neha, Mumika, Neha, Mira,

[1:57:01] list as well as through dictionary. Do you see this? This can be created with a specified index or list as well as through dictionary. Getting my point, learners?

[1:57:25] access any of the elements, default indexing is also there, that is 0 1 2 3 indexing is also there, that is 0 1 2 3 4 and 5. So, it says to get the value

[1:57:38] two, so it will give me the value three, or I can access only the element for B. or I can access only the element for B. Clear?

[1:58:01] Okay. This was print series. Yeah.

[1:58:16] pass the index, please try to understand over here, learner. Just add a print series over here. If I do not pass the date index, it will by default give me 0 1 2 3. And if I pass it with index, then it

[1:58:30] will use the index value. Clear?

[1:58:51] functions in the series. Right? In the pandas series. These functions collectively help analysts summarize and understand the

[1:59:03] characteristics of the data facilitating effective data exploration and data analysis. So, please try to understand these functions. Everybody to be concentrated over here. Very, very important

[1:59:17] analysis. The first is this head function which gives the first n rows. If I have passed the parameter three, it gives me the first three. If I pass it as 13, it gives me the first 13 rows. And if I do

[1:59:34] gives me the first 13 rows. And if I do not pass any parameter, default value is equal to five. Okay? Okay? Then the default value is equal to five.

[1:59:46] Tail is the last three rows. Shape gives me the dimension, rows of column. Of course, when we are talking about series, series will always be

[1:59:58] about series, series will always be one-dimensional array. Agreed, learners? one-dimensional array. Agreed, learners? Series will always be one-dimensional When I move ahead to the describe function, it gives me the statistical

[2:00:12] analysis. Uniques where it gives me the unique value of the function. Please use these functions. And n unique gives me the number of unique values.

[2:00:27] Got it, learners? So, it doesn't print anything. So, we can print one of them. Uh

[2:00:39] That's why I keep making changes in the file. file. So, it becomes first.

[2:00:53] rows? And if I change it to 13, since there are only five elements, it will give me five. And the moment I change it to one, it gives me only one element. to one, it gives me only one element. Clear?

[2:01:15] n unique gives me the number of unique values.

[2:01:29] Clear? Vebhav, are you able to do it? Neha, Bhumika? Now, I've just broken those information and seen the output. Just simple functions. Got it?

[2:01:50] transformations in pandas series. Operations and transformation in pandas series are crucial for modifying and enhancing. All these operations help in cleaning of the data. So, if I want to see the element by

[2:02:05] So, if I want to see the element by element addition,

[2:02:19] if the um the output. Yeah? So, 0 1 2 3, and then the other one is

[2:02:32] a b c d. None of them are common, so I get the output as n a n. What does n a n Not uh number. Then I am doing the squared of the values. Again, I'm just doing just

[2:02:48] copy-paste, cutting it, and just checking out the output.

[2:03:00] become my one, right? So, wherever the value one was there in this series, let value one was there in this series, let me get back to that series.

[2:03:13] two gets number two, and three gets number three. number three. Clear?

[2:03:27] Similarly, by using sort_values, we can sort the values, also. to be missed over here. Just to check on the null values,

[2:03:42] you can use the null. So, this is These are the functions which give power to are the functions which give power to the pandas library for the analysis. So, they aren't any missing values. So, the answer is going to be false,

[2:03:56] So, the answer is going to be false, false, false. are any missing values, null value, it will lot of um

[2:04:11] you know, practical you know, data with this, so nothing to there with me? Got it? So, just to recap, you know, basically, you can do element-wise addition in

[2:04:25] series, apply to apply each of the function mapping using dictionary function mapping using dictionary values, sort_values to sort the data, and to check the null value is the is null function.

[2:04:38] Are we good to go? Can we move on to the next file? And not the next file, next topic, that is querying a series. So, if this is my

[2:04:50] is querying a series. So, if this is my series, right, over here. 30, it will give me only those values greater than 30, lesser than 30.

[2:05:03] Right? So, these are the outputs. So, this is my series, the original series, and I want to see only the values greater than 30, I just only the values greater than 30, I just have to apply the conditional operator.

[2:05:20] So, series indexing, only those values greater than 30 get printed. Values only greater than 30 get printed. Values only equal to 20 get printed. Values not equal to 40 get printed. Values greater than 20. And And operator is also clear?

[2:05:38] Lesser than 50 get printed. Right? And is in. So, lots and lots of practice needs to be done. For the next session, you need to For the next session, you need to complete all this file before we meet in

[2:05:52] the next session, right? And finally, and not the least, you have the assisted practice to be done for the series function. And in this whole journey of data science, as I've been telling you,

[2:06:06] statistics is going to be a very, very important part. And when I talk about the term descriptive statistics, please try to understand. When I talk about the term descriptive statistics, it's all about describing about the data. Right?

[2:06:24] What do we want to describe about the data? The mean, the standard deviation, the minimum, the maximum, and the Q1, Q2, Q3. Q1, Q2, Q3 refers to the

[2:06:36] Q2, Q3. Q1, Q2, Q3 refers to the quartiles of the data. Q1 refers to the 25% of the data. Q2 refers to the 50% of the data and Q3 refers to the 75% of the data.

[2:06:51] Right? So, now if I look at the first concept, mean we are very clear. How do we calculate mean? We take the total sum of the value divided by the total number of the value, the number of the values that is there.

[2:07:07] When we talk about standard deviation, standard deviation is a method for standard deviation is a method for calculating how far are we away from the mean point. So, the mu is represented mu represents the mean of the data.

[2:07:22] And sigma is the standard deviation that how far are we away from this particular mean point. Clear? Right? We will delve delve into more of statistics, but try to understand that's

[2:07:37] the mathematical formula. And if I square the value, the standard deviation becomes my variance. What is the relationship between standard deviation and variance? That the square of standard deviation is known as the

[2:07:53] variance of the data. Right? Right? Do we understand the relationship between standard deviation and variance? Then coming on to the quartiles, when we divide the 100% of the data into

[2:08:08] four parts, Q1 refers to the first percentile or the 25th percentile of the percentile or the 25th percentile of the data. Q2 refers to the 50th percentile or the mid-range of the data. And Q3 refers to the 75th percentile of the

[2:08:24] data. Interquartile range, also known as the Interquartile range, also known as the IQR. That is equal to Q3 minus Q1.

[2:08:36] you have to just need to understand the formula. Rest everything will be done by the Python function that refers to the quartile. So, when I do when I have

[2:08:48] these values, I've created a series with two, three, four, and I run this of this describe function is gives me the count, the number of values, the mean value, the standard deviation, minima, Q1, Q2, Q3, and maximum value.

[2:09:06] minima, Q1, Q2, Q3, and maximum value. So, I get all the values in one single Clear? And similarly, if I have categorical or string values, the describe function gives me the count. It has four letters.

[2:09:20] gives me the count. It has four letters. Unique values are three. Top most value is P, and P has been repeated two times. P has the highest frequency two times. Now, moving ahead to the next concept today, that is data frame. Data frame is

[2:09:38] nothing but a two-dimensional data structure. That is data is aligned in a tabular fashion in terms of rows and column. So, data frame is nothing but a two-dimensional data structure. That is

[2:09:53] the data is aligned in tabular fashion of rows and column. Data frames are you know, are of different types because every column is different. It could be

[2:10:05] one of them could be int, float, or string type. Definitely, data frames are mutable. They have the labeled axis in terms of rows and column. And it can terms of rows and column. And it can perform arithmetic operations on rows

[2:10:19] and columns, also. Getting my point? So, if we talk about the concepts of data frame, we the concepts of data frame, we understand access zero refers to the

[2:10:31] rows. Please be clear. Access zero refers to the rows, and the values can refers to the rows, and the values can be accessed via df.index.

[2:10:46] Okay? And columns refer to access one, that is represented as df.columns. And all the values in the table are can be accessed with the help of df.value. So, the first way to create, uh you know, data frames is Can you tell me

[2:11:03] what is the data structure? The first way to create data frame is using dictionary. Learners, are you there with me? The first way to create dictionary is you creating data frame is using dictionary.

[2:11:18] First thing, import the pandas as pd, right? Secondly, the key values are automatically become my column heading, and these are my list or the data

[2:11:30] points. Getting my point? This is nothing but my data frame, pd.data Everybody is able to run the code. I've just split the code so that it is easier

[2:11:42] for us to understand the different ways to create data frames. Create, uh you know, data frame is using the list of list.

[2:11:55] Yeah. Are you getting this? So, this is pd.data frame. This is the data list, and this is the column list. Are you understanding the different ways to create data frames, learners? Are you able to run the code along with

[2:12:11] me? The way to create is array. What is What What is the dimension of array that we are using over here?

[2:12:28] are using over here? What are the What is What is What is the dimension of this particular array? Quickly tell me. np.array creates an array. What is the dimension of this array? Two Two square

[2:12:42] brackets, what do they represent? Yes, it's a two-dimensional array, which Yes, it's a two-dimensional array, which helps us to create the data frame. That's a tabular structure. And another way is by reading a CSV. Should I share

[2:12:59] the CSV file with you? I'm sharing the CSV and Excel file. Please download it and copy it into the same folder. So, when I load this, this will run. It will not run because it's not there. So,

[2:13:15] will not run because it's not there. So, let me see. You'll find all these files in the data set folder.

[2:13:45] be present in the same folder. So, how do we go about it?

[2:14:07] Do you see I've already copied the CSV file in here? And now if I run my code, it's able to give me the output. Do you see that, learners?

[2:14:27] this house prices.csv? Only Sahana is able to do that. What Only Sahana is able to do that. What about others?

[2:14:46] So, data frames can be created from dictionary, list of list, NumPy array, reading the CSV file, reading the Excel file. Right? And when I read this Excel file now, the iris.xlsx,

[2:15:00] now, the iris.xlsx, that's how I get the output. Anybody with a thumbs up thumbs down who's not able to run this code, please

[2:15:15] let me know. Data frames is getting clear to And another way that we have understood is using a dictionary. The keys become my column heading. And it is the head function which gives me the top end

[2:15:29] values. Got it? Right? the sorts of indexing available in basic Python or NumPy? Anybody who remembers that what are the

[2:15:45] two types of in basic Python you know, Python or NumPy? Nobody understands basic indexing? Yes, learners, I'm waiting for a response. It refers to positive indexing as well

[2:16:02] It refers to positive indexing as well as negative indexing. Please look here. What What does it mean? We We move ahead with positive indexing as well as negative thing. But, Pandas library is different. Right? Here, we can define

[2:16:18] our own labels. So, how do we give the indexing part of it? How do we give the indexing part of it? How do we give the indexing part of it? Tell me quickly. That's known as label indexing

[2:16:33] represented as {dot} loc and integer-based integer-based or position-based indexing known as iloc. Do you remember that point in series or not?

[2:16:47] Yes, learners. NMIT students, are you there or not? So, label indexing refers to the label or the values that we have given. Column,

[2:16:59] the values that we have given. Column, value, index, they refer to the {dot} loc values. Row index values refer to loc. Right? And by default, even position-based integer indexing are uh present that can

[2:17:16] be accessed via iloc. Got my point? Got my point? Getting my point? Explaining this with further example, let me move ahead that indexing in Pandas can be label indexing

[2:17:32] or position-based. Right? Label is what label we have explicitly given to the rows or columns. Definitely, columns have name, but if they are no names no no names given to the rows, it will take

[2:17:47] 0 1 2. Okay? Do you remember this type of indexing? This type of indexing is used to extract multiple values. This type of indexing is known as list indexing. If I want to

[2:18:03] two. And another important point, slicing. Slicing is extracting the part of the number of columns, right? So, here we

[2:18:15] have zero to two, right? Over here. But, another major difference between label and position-based indexing is that in label-based indexing indexing is that in label-based indexing zero and two both are included. Whereas

[2:18:30] in position-based, this two is not in position-based, this two is not included as in normal and basic Python. Sahana, what do I mean over here is that slicing over here means that zero to

[2:18:44] two, the labels that we have given to the column. But, in this case, zero and two both are included, but in integer-based or position-based, it's like the normal slicing. Only and zero and one are included. Two is not

[2:18:59] included. To make the concept even more clear, let me Let's understand with an example. So, if this is my data frame, the first bracket, please try to understand this. So, the first one is known as for the

[2:19:17] So, the first one is known as for the indexing. The first square bracket is used for indexing, and the second square bracket is used for the list. Getting my point? Indexing. So, what does it do? That it

[2:19:33] extracts the two columns automatically. That is employee ID. If I want to extract age, that I can also add. And if I say integer-based indexing with

[2:19:45] And if I say integer-based indexing with zero at value, it refers to the first zero at value, it refers to the first row because here I am, you know, Can you tell me data frame? Data frame is like two dimension

[2:20:03] if there is no {comma} given, so it becomes row {comma} c. Getting my point?

[2:20:15] Right? But again, if I want to extract multiple rows, again, I can use the concept of list indexing also. Now getting clear, Sahana, Aishwarya, Neha?

[2:20:30] Neha? Nayana? And do you remember this concept now, learners? Do you remember this concept? This is label-based indexing. If there are two colons used, what is this known

[2:20:45] as? If there are two colons used, what is this known as? Anybody who remembers the concept? I explained this concept is concept? I explained this concept is known as striding.

[2:20:59] It's not string, it's striding, okay? So, the last is the step parameter, right? We start from the zeroth row, right? And we will jump two values. So,

[2:21:12] what are the rows that we can get? We get zeroth, second, and the fourth row. get zeroth, second, and the fourth row. Got my point, learners?

[2:21:28] refers to the rows, and this is after the {comma}, it refers to the column. So, over here, what is the idea? That the employee ID is also included, the scale is also included, as well as the age is included. This is another

[2:21:44] difference. I hope this example in the PPT is making the concept clear. possible elements in pandas library is based on two indexing. One is known as

[2:22:00] label indexing and another way is known as integer based indexing. Can you tell me how are we creating the data frame over here?

[2:22:12] How are we creating the data frame over here learners? Which data structures are we using? What is data over here? Quickly tell me.

[2:22:26] I'm just separating the code. If you want you can do that, but so that that gives me, you know, a little more better easier thing or if I remove this, this is how I get the data frame.

[2:22:39] Clear? Yes learners, are you there? Everybody is able to run the code? How can I access a single column? Just by giving the column name, it prints me the column data. If I want to access multiple

[2:22:56] columns, I can use the concept of list indexing. Clear? Difference between accessing single column and multiple columns is clear? Everybody is able to get this point?

[2:23:11] Just by single column name and if there are multiple, then I can add the list onto it. Do we understand iloc learners? What is Do we understand iloc learners? What is iloc? Integer based indexing where I

[2:23:25] iloc? Integer based indexing where I access the specific rows. If I want to access the rows based on condition, I'm I just do it because I think so it's easier to run the code and get the output rather than first see the

[2:23:40] scrolling. That's why I'm just cutting the code and checking giving you the output. So only those columns get you know, get printed with column name greater than 15.

[2:23:54] greater than 15. Got it, learners? label as using is at. This is the zeroth column and the column name or we can

[2:24:06] column and the column name or we can also give the index value. Both will give the same thing or we can also use the label index. So, all the I think so the label index. So, all the I think so three will give me the same output.

[2:24:23] This is five. This is 10. This is by label and this is LOC. So, why the position is not giving me the output? Position is not giving me the output because if I give zero, then I will get all the same values.

[2:24:38] So, you would say that ma'am, what is the use of label or integer base? They are exactly same methods but different ways to access the data. It's completely

[2:24:50] up to you. You want to use the index position or the label position. Everybody is has been able to run this code or has been able to run this code or Waiting for the response.

[2:25:05] Abhijita, you able to do it? That's great. What about others? Are you all there in the lab? 100 plus. Oh my god, you got it. 120 now. Come on, learners. Respond at least 50% of you respond.

[2:25:18] Manvith, Mania, Maruti, Meera, Monica. Quick, quick, quick. Prajwal, Prakhyat, Pragna, Poornesh. Come on, respond everybody. Shreya Come on, respond everybody. Shreya Shivaraj, Shashank, Shashwat

[2:25:33] Shivaraj, Shashank, Shashwat Okay, great. basics, the commands remain the same whether we are talking about um

[2:25:45] whether we are talking about um the series or data frame. The head and tail methods enable users to efficiently preview the initial and the final rows of the data frame, offering a quick snapshot of its

[2:25:59] structure and content. These functions are invaluable for preliminary are invaluable for preliminary assessment. So, now this is the power of Pandas library. When I say that Python is a good language, supports simple

[2:26:14] function, so this is where the power of Pandas library head function. Please be clear with these functions, everybody. It returns the first few rows. Tail gives me the last few rows. Info provides me the summary of the data. I

[2:26:30] asked about describe, it gives me the statistical summary in terms of count, mean, standard deviation, minimum, maximum, and quartile value. Then we move on to the shape, which gives me the number of rows and column in the data

[2:26:47] frame. And without these functions, no analysis is complete. Column gives me the returns the column labels. LOC and I LOC are the way in which we can achieve

[2:27:00] indexing and slicing of rows and column. Sort underscore value will sort the values. Group by will group them. Apply applies each function to each element. Merge and concatenate will merge the

[2:27:14] data set. Plot will plot. And drop removes the specified row or column. So, we will as we move along, we will explore these functions. In-place function, how will you override the existing data frame? So, do you want

[2:27:31] to override the existing data frame? If yes, then we will always make the in place parameter equal to true. And if you do not want to make the changes permanent, then we will make in place is equal to false. So, that we will see

[2:27:48] once we do it practically. But before that, we need to understand correlation. Another very important aspect of statistics. Please try to understand.

[2:28:00] statistics. Please try to understand. This is the formula for R, that is Pearson's correlation coefficient. What does it help us to do? It helps us to find out the relationship between two to three variables. So, Karl Pearson's

[2:28:14] coefficient of correlation, where value ranges between -1 to +1, which gives tries to give the relationship between X and Y. So, what

[2:28:28] relationship between X and Y. So, what does -1 represent? -1 if the R value of does -1 represent? -1 if the R value of the R is -1, it it shows a strong negative correlation between X and Y.

[2:28:40] What does negative relationship mean? That if X increases, Y decreases. That if X increases, Y decreases. All right? Or if Y increases, X

[2:28:52] All right? Or if Y increases, X decreases. That's you know, that's the negative correlation. And when R is equal to +1, what is the positive equal to +1, what is the positive correlation mean? X increases, Y also

[2:29:06] correlation mean? X increases, Y also increases. Or X decreases, Y also decreases. Getting my point? That if X decreases, then Y also

[2:29:19] That if X decreases, then Y also decreases. Getting my point? everybody? And when R is equal to zero, that means there is no relationship between the data points.

[2:29:35] data points. Got it, learners? Very, very important to understand the in-depth of the column. For example, for example, if this is the data that has been loaded for your maths maths, physics, and

[2:29:50] giving. Simply, we have loaded the CSV file or the Excel file containing the marks of maths, physics, and history. And when I run the function df.corr,

[2:30:06] it tries to create an n by n matrix. Please try to understand. Since there are three columns, so a 3 by 3 matrix is created. What is the use? It gives me relationship between maths and maths. Of

[2:30:21] course, diagonal elements is going to be one because we share positive relationship with ourselves, right? We are perfect with ourselves. So, there is this perfect relationship between one to one, right? But what is the relationship

[2:30:38] between maths and physics? It's very strong. It's very close to 1.9. But the relationship between maths and history is negative with each other. Similarly, if we talk about history and physics, they also share a

[2:30:56] history and physics, they also share a negative relationship with each other. Getting my point? Is the correlation getting clear to everybody?

[2:31:09] correlation? Sahana, Meera, Nagendra, Naina, Neha, Then there is this another function value_counts,

[2:31:25] which helps us to calculate the unique value. So, based on the categories, they value. So, based on the categories, they are north, east, west, and south. How many times east has been repeated? Only two times. How many times north? One.

[2:31:40] How many times south? One. And west was two times. two times. Got it, learners? everybody? So, getting back to the practical file.

[2:31:57] So, which is the Python function which gives the correlation between the values? Yes, learners. Which is the Python function which gives me the correlation between the values? It is df.corr. And what is the range of

[2:32:12] the values? The range of the value is always between -1 to +1. Here, they all always between -1 to +1. Here, they all share a positive relationship. But if I want to make it to negative, yes, Akshat, there is some issue. Yeah, it's

[2:32:27] df.corr. And if I really want it uh to be uh negative, maybe I can put a negative sign over here. Then, run and see the output. So, between column one and column two,

[2:32:44] there's a negative relationship. But one and three uh you know, share uh positive relationship. Two and again, two and three share a perfect uh negative relationship. And that is true,

[2:32:58] also. Clear? counts, which gives me the categorical data along with the frequency value.

[2:33:10] Are we clear with this particular point? Are the concepts clear? Are we able to run the code? And this is the assisted practice for you all that you have to load the housing data set and then perform such

[2:33:26] actions on it. Will you be able to do that? This is the data set. You can save it for your homework assignment. Great. So, what is the use of date time

[2:33:39] function in any coding language? Can anybody tell me? What is the use of date anybody tell me? What is the use of date time function in any coding language?

[2:33:55] Now, look at over here. I am creating this. Do we have any date type over here? What is the date today? Let me put that as is the date today? Let me put that as that. 04

[2:34:15] All right? Now, suppose if I don't give the code, okay?

[2:34:27] uh even print this. Suppose it's it's saying that no literals are allowed. It doesn't allow me to give three. So, let me remove this. So, it works. Let's see.

[2:34:44] 2025 mean? We are not even getting the correct answer.

[2:35:00] So, that is why a library has been built in Python to deal with date time in Python to deal with date time objects.

[2:35:18] Right? Got it, learners? And if I do a date do you think I'm getting the correct Do you think I'm getting the correct answer? No.

[2:35:31] answer? No. Right? So, that is why to deal with date time, most of the time the real-time projects will contain columns which are, you know, of date and time. It could be arrival, departure of the flight, when

[2:35:46] that particular recording was done, how long the session went on, right? And who bought at what particular time, when the items were sold, whenever they were items were sold, whenever they were shipped. Every real-time data requires

[2:36:01] shipped. Every real-time data requires this date time analysis. Got my point? So, in Pandas, the date time module provides robust functionality for handling date and time. So, Pandas library, because Pandas is a library

[2:36:16] which helps in data manipulation, real-time data. So, in Pandas, the date time module provides robust functionality for handling date and time

[2:36:28] data, while the time delta class allows us for convenient manipulation of the time interval. This combination is particular particularly useful for time-based analysis and working with temporal data

[2:36:44] in the data frame, right? So, if I want to access what is the date time now, to access what is the date time now, then what is important over here is that then what is important over here is that we need to import the date time module.

[2:36:59] The first thing over here, please try to code along with me, learners. Import date time. Okay? And if I say I'll just send the code today is equal

[2:37:15] to date time dot now. data. So, it doesn't have now or if let's see

[2:37:29] then. It's not working. Let me see. It's not working. Let me see. So, DT, sorry. This is the accessor and then we have this as date time dot

[2:37:43] today. Now, do you see? It gives me today is uh you know, um what is it? 4th of March 2026. And if I want the time to be now, how will I go about it?

[2:37:56] DT dot dot now.

[2:38:10] calculations on that, then I can use this time delta function. For example, this time delta function. For example, if today's date was 26th June, it would have added three six days to that to get me this output.

[2:38:25] Clear? Rather five days. It is actually five days, so there is this error. So, it is five days which can be added. We'll just Getting my point, learners? Are you there with me? So, that's why data

[2:38:41] handling is very, very important because it involves handling of the real-time data set. So, first thing, how do we create a date range in pandas? So, the

[2:38:54] date range function is used to generate a sequence of dates with a specified range. So, we can generate sequence of dates. It's a powerful tool for creating

[2:39:06] time and indexes or date column. For example, if I want to create dates For example, if I want to create dates starting from 1st of January 2023 to starting from 1st of January 2023 to 10th of January daily, I can pass this

[2:39:20] as frequency equal to D. session. I hope so you would have got a brief idea regarding data analytics. Thank you guys for watching this video. Subscribe Simply Learn for more

[2:39:33] Subscribe Simply Learn for more successful courses.