AI Summary
The video is a comprehensive but fragmented course on agentic AI, covering fundamental concepts like generative AI and large language models, as well as more advanced topics like AI agents, workflows, and tools. However, it is presented as a series of disjointed tutorial introductions rather than a cohesive, in-depth educational experience, with significant portions devoted to promotional content.
Agentic AI is defined as an AI system that can understand goals, plan tasks, and complete complex workflows with minimal human intervention. It is changing how we interact with technology by automating processes and building intelligent assistants.
Generative AI refers to AI designed to create new content, such as text, images, and music, based on patterns learned from vast data sets. Tools like ChatGPT and DALL-E are examples, and its applications range from content creation to healthcare simulations.
Large language models (LLMs) are sophisticated AI systems trained on vast datasets using deep learning and transformer architectures to understand and generate human-like text. Their applications include powering virtual assistants, chatbots, content creation, and language translation.
AI agents are advanced virtual assistants designed to perform tasks autonomously. They automate repetitive tasks, enhance decision-making with data-driven insights, and act as personalized digital assistants across various tools and apps.
OpenAI released two groundbreaking agents: Operator, which mimics human actions on a browser using a new CUA model, and Deep Research, which performs comprehensive research and compiles information into detailed reports using the O3 model.
Agentic AI works in four steps: perceiving (gathering data from databases, sensors, APIs), reasoning (using LLMs to interpret tasks and generate solutions), acting (executing tasks autonomously via external tools), and learning (improving through feedback loops or data flywheels).
Agentic AI differs from generative AI by not just generating responses but also executing tasks autonomously. It is adaptive, learning from its actions, and requires minimal human involvement, unlike generative AI which relies on user prompts.
MCP (Model Context Protocol) is an open-source protocol that standardizes how applications supply context to large language models, simplifying the integration of AI models with various data sources and tools.
Mentioned in this Video
Tutorial Checklist
💡 Key Takeaways
How Generative AI Creates Images
Provides a clear, step-by-step breakdown of the image generation process, including data collection, neural networks, tokens, and feedback, making a complex process accessible.
05:58OpenAI's New Agents: Operator and Deep Research
Highlights the release of two groundbreaking agents, Operator and Deep Research, explaining their unique capabilities and the new CUA model, which is significant for AI agent development.
33:19Agentic AI Works in Four Steps
Defines the core operational cycle of agentic AI (perceive, reason, act, learn), which is fundamental to understanding how autonomous AI systems function.
44:42What is Generative AI?
Gives a clear, accessible definition of generative AI, contrasting it with traditional AI and providing a foundational understanding for beginners.
47:42Full Transcript
[00:08] Imagine an AI system that could understand your goals, plan tasks, make complete complex workflows with minimal human intervention. Sounds interesting, right? Well, that's exactly what agentic AI is
[00:22] all about. From automating repetitive business processes to building intelligent assistants, analyzing data, and creating AI-powered applications, agentic AI is changing how we interact with technology. Businesses are
[00:34] exploring ways to use AI agents to improve productivity, automate workflows, and solve problems that traditionally require multiple manual steps. But here's the big question. How do these AI agents actually work, and
[00:47] how can you build one yourself? Hey everyone, welcome to this agentic AI full course by Simply Learn. In this comprehensive course, we'll take you through the fascinating world of AI agents, starting with the fundamentals
[01:00] applications. You'll begin by understanding what AI agents are, how they interact with different environments, and how agentic AI workflows operate. We'll also explore essential concepts like natural language
[01:14] processing, generative AI, reinforcement learning, and Q-learning. As we move forward, you'll learn how to build AI agents from scratch, create AI systems that combine information retrieval with intelligent AI workflows.
[01:29] platforms like LangChain, LangGraph, LangFlow, LangSmith, Hugging Face, and roles in developing AI-powered solutions. You'll also explore AI tools and models, including Manifold AI, Deep Seek, Llama, Claude, and Gemini, along
[01:46] with practical topics like Python-based AI web scraping, AI video generation, and generative AI for data analysis. We'll discuss career opportunities, the generative AI engineer roadmap, and ways AI agents can create business value.
[02:01] Whether you're a beginner exploring AI or a developer looking to expand your skills, this course will help you understand the concepts, tools, and workflows behind agentic AI. That said, if these are the types of videos you
[02:14] subscribe button along with the bell icon to get notified whenever we upload a new video. Before we start, if you're looking to build deeper expertise in agentic AI and learn how autonomous AI systems are designed for real-world use,
[02:29] here's something worth checking out. The Microsoft Applied Agentic AI Systems Design and Impact Program is designed for professionals who want to move beyond basic generative AI and learn how to build and manage
[02:43] production-ready AI agents and multi-agent systems. Through live practitioner-led sessions, hands-on projects, guided practices, and a capstone project, you'll build practical experience with agent design,
[02:57] orchestration, rag, MCP, planning systems, workflow automation, and The program also includes a joint completion certificate from Microsoft and Simply Learn, along with Microsoft Learn credentials and Microsoft branded
[03:12] agentic AI learning content. Practical use cases involving agentic rag systems, multi-agent AI-powered automation, agentic UX, financial analysis, and production-grade go-to-market systems helping you understand how autonomous AI
[03:27] solutions are actually designed and deployed. So, if you want to build production-ready agentic AI skills and gain practical experience creating autonomous AI systems for real business scenarios,
[03:40] check out the link in the description box below and in the pinned comments. Now, let's get started. What is the primary purpose of an AI agent? A, to store large amounts of data without processing it.
[03:54] B, to perceive its environment, make decisions, and take actions toward achieving specific goals. C, to perform only mathematical calculations without interacting with external systems. D, to replace all programming languages with
[04:06] natural language. Let us know your answer in the comment section below. Let's get started with this course. >> Meet Emma, a graphic designer working on a new project. One day, her colleague mentions a tool that helps create
[04:19] mentions a tool that helps create designs, images, and text using AI. Intrigued, Emma wonders how AI can create something from scratch. Her curiosity grows, and she decides to dive deeper into this new technology
[04:33] called generative AI. Generative AI refers to a type of artificial intelligence designed to create new content, such as text, images, music, and videos. Unlike traditional AI, which analyzes or
[04:46] categorizes data, generative AI produces original content based on patterns learned from vast data sets. Essentially, it generates new, unique material. These models are often trained on large
[05:00] amounts of data and use sophisticated algorithms to mimic human creativity. Tools like ChatGPT or DALL-E can create art, write essays, or simulate conversations by generating outputs based on user prompts.
[05:14] Generative AI has a wide range of applications. Content creation. Tools like GPT-4 generate text, blog posts, stories, and essays from simple Art and design. AI models such as DALL-E generates
[05:29] unique images and designs based on text descriptions, transforming creativity in Music and audio. AI can compose music or replicate voices, offering new possibilities for musicians and audio engineers.
[05:44] Generative AI simulates disease progression or create synthetic medical data, helping doctors gain faster insights for research. Let's take image generation as an example to explain how generative AI
[05:58] works. Data collection and learning. AI models like DALL-E are trained on large data sets of images paired with text descriptions. These data sets teach the model to recognize different
[06:10] objects, colors, styles, and how to associate text with corresponding The more data the AI learns from, the better it can generate accurate and better it can generate accurate and diverse images based on user prompts.
[06:23] Neural networks and transformers. When Emma inputs a prompt like a cat wearing sunglasses, the transformer model processes the text, recognizing words like cat and sunglasses, and links them to images it learned from during
[06:36] training. Transformers help the AI decide how to image. Tokens and context. The text input, such as a cat wearing sunglasses, is split into smaller parts
[06:50] called tokens. The AI processes each token and understands the relationship. For instance, it knows the sunglasses should be placed on the cat, creating a contextually accurate image. Feedback mechanism.
[07:04] Generative AI models improve through feedback. After generating an image, users provide feedback on the accuracy or quality of the output. If Emma's generated image shows the sunglasses floating beside the cat, she can mark it
[07:18] The model uses this feedback to improve future image generations. Reinforcement learning further enhances The model's rewarded when it generates accurate images and corrected when it
[07:33] For example, when Emma describes a sunset and the AI produces a vibrant sunset image, it receives positive reinforcement. model's ability to generate better images.
[07:48] Data science and AI models. Data scientists curate the training data and define the parameters that help the AI generate accurate images. versatile the AI becomes in generating diverse types of content.
[08:02] parameters, which are settings that guide the AI in processing data and generating outputs. Generating original content. Once trained, the model can generate original images. For example, you then
[08:16] might describe a futuristic cityscape, and the AI would produce a unique image based on what it learned. The generated image isn't just a copy of past data, but an entirely new creation, showcasing the AI's ability to combine
[08:30] learned patterns and creativity. >> Large language models, LLMs. If you ever wondered how machine learning can now understand and generate human-like text, chatbots like ChatGPT to AI assistant that power search engines, LLMs are
[08:45] transforming how we interact with technology. One of the most exciting advancement in this space is Google's Gemini or OpenAI's ChatGPT, a designed to push the boundaries of what AI can achieve. In this video, we will
[08:59] explore what LLMs are, how they work, and why models like Gemini are critical for the future of AI. Google Gemini is part of a new wave of AI models that are smarter, faster, and more efficient. It is designed to understand context
[09:14] better, offer more accurate responses, and integrate deeply into service like Google Search and Google Assistant, providing more human-like interactions. behind LLMs, including their massive training data set, transformer
[09:28] architecture, and how models like Gemini use deep learning innovation to change industries. Plus, we will compare Google Gemini to other popular LLMs, such as open AI chat GPT models showing how each of these technologies is used to power
[09:41] chatbots, virtual assistants, and other AI driven applications. By end of this understanding of how large language models like Gemini work, their key features, and what they mean for the
[09:53] subscribe, and hit the bell icon to never miss any update from Simply Learn. So, without any further ado, let's get started. So, what are the large language models? Large language models like chat GPT 4 generate a pre-trained transformer
[10:06] 4 0 and Google Gemini are sophisticated AI systems designed to comprehend and are built using deep learning techniques and are trained on vast data set leverage self-attention mechanism to analyze relationship between words or
[10:23] tokens allowing them to capture context and produce coherent relevant responses. LLMs have significant application including powering virtual assistant, chatbots, content creation, language translation, and supporting research and
[10:35] decision making. Their ability to generate fluent and contextually language processing and improved human-computer interaction. So, now let's see what are large language model used for. Large language models are
[10:48] utilized in scenarios with limited or no domain specific data available for training. These scenarios include both few-shot and zero-shot training approaches which rely on the model's strong inductive bias and its capability
[11:00] to derive meaningful representation from a small amount of data or even no data at all. So, now let's see how are large language model trained. Large language models typically undergo pre-training on a boat all encompassing data set that
[11:14] shares statistical similarities with the data set is specific to the target task. The objective of pre-training is to enable the model to acquire high-level feature that can later be applied during the fine-tuning phase for specific task.
[11:27] So, there are some training processes of LLM which involves several steps. The first one is text pre-processing. The textual data is transformed into a model can effectively process. This conversion may be involved techniques
[11:40] like tokenization, encoding, and creating input sequences. The second one model's parameter are initialized randomly before the training process begins. The third one is input numerical data. The numerical representation of
[11:53] processing. The model's architecture, typically based on transformers, allows it to capture the contextual relationship between the words or tokens function calculation. A loss function calculation measures the discrepancy
[12:07] between the model's prediction and the actual next word or token in a sentence. The LLM model aims to minimize this loss during training. The fifth one is parameter are adjusted through optimization technique. This involves
[12:21] parameters accordingly, gradually improving the model's performance. The last one is iterative training. The training process is repeated over multiple iterations or epochs until the model's output achieve a satisfactory
[12:34] level of accuracy on the given task or data set. By following this training process, large language models learn to capture linguistic patterns, understand responses, enabling them to excel at various language-related tasks. The next
[12:48] topic is how do large language models work? So, large language models leverage deep neural network to generate output based on patterns learned from the language model adopts a transformer architecture, which enables the model to
[13:01] identify relationship between words in a sentence irrespective of their position in the sequence. In contrast to RNNs that rely on recurrence to capture token relationship, transformer neural network employs self-attention as their primary
[13:15] attention scores that determine the importance of each token with respect to the other token in the text sequence, facilitating the modeling of intricate relationship within the data. Next, let's see application of large language
[13:29] wide range of application across various domains. So, here are some notable language processing, NLP. Large language language understanding tasks such as sentiment analysis, named entity
[13:43] recognition, text classification, and language modeling. The second one is chatbot and virtual assistant. Large language models power conversational agents, chatbots, and virtual assistant, providing more interactive and
[13:55] human-like user interaction. The third one is machine translation. Large language models have been used for automatic language translation, enabling text translation between different languages with improved accuracy. The
[14:07] fourth one is sentiment analysis. LLMs can analyze and classify the sentiment or emotion expressed in a piece of text, which is valuable for market research, analysis. The fifth one is content recommendation. These models can be
[14:21] employed to provide personalized content recommendations, enhancing user experience and engagement on platforms such as news website or streaming highlight the potential impact of large language models in various domains for
[14:34] automation, and interaction between humans and computers. >> Now, the agenda for today's session. Firstly, let's get started with the agenda for today's session. Now, we will have a brief introduction
[14:46] to this session. Followed by that, we will have a briefing of what exactly is AI, then how does it work. Next, we will understand the key technologies and frameworks involved, and we will also go for some everyday use cases where we can
[14:59] take help of AI, and also some of the responsibilities that we need to keep in mind for the ethical use of AI, and lastly, we will explore some prompts. the agenda, let's get started with the first part, introduction.
[15:16] How can AI simplify daily tasks? As we discussed in the use case, a human might have a lot of routine and mundane tasks. For example, automation. AI can simplify your daily tasks like checking up emails and drafting email responses, enhancing
[15:32] productivity by getting AI assistance for research and marketing trends or any other researches based on your job profile. And you can have a personalized AI assistance for recommendations for learning and much more. And after that,
[15:46] it will save you a lot of time reducing effort. Now, Get set up with generative AI models like Microsoft Copilot and try some of the examples. So, if you get access to one of the generative AI models like
[16:00] GitHub, then you can access GitHub Copilot easily, use it in Microsoft 365 apps like Word, Excel, and Teams. Then you can also explore prompting basics to start up with simple commands like drafting emails or summarizing
[16:13] documents. And proceeding ahead, we have another opportunity to experiment with different use cases like trying Copilot with brainstorming, planning, and organization. And fourth one, customize and iterate. Improvise AI-generated
[16:26] results by refining prompts based on your needs and requirements. Now, let's proceed ahead and understand what exactly is AI and how it works, which will involve some of the key topics like understanding the basics and
[16:39] technologies involved, etc. So, generative AI is an advanced type of artificial intelligence that can sync up with human activities and gather requirements to assist humans in their daily tasks, improving efficiency and
[16:53] effectiveness. Not just these professional tasks, AI can create new content like text, images, music, and videos. It uses natural language processing and neural networks to learn patterns from existing data and then
[17:06] patterns from existing data and then generate new content. Now, let's see how we can use AI to simplify our daily tasks and make life easier. Firstly, AI can create unique content, not just analyze data and make
[17:20] predictions. It can assist you in real-time generation of original text, images, music, and more. Next is AI as a creative partner. Generative AI can and much more. It goes beyond traditional AI. Unlike
[17:36] predictive AI, generative AI creates new content rather than just analyzing data. functionality where it can enhance human creativity by assisting with brainstorming ideas, storytelling, design, and content generation. Lastly,
[17:51] it is widely used in multiple fields, found in marketing, education, entertainment, and software development. Now, let's proceed ahead and understand the key technologies behind generative AI. Large language models is the first
[18:04] one. Trained on vast amounts of text to generate human-like responses. For example, ChatGPT, PaLM, LaMDA, and Microsoft Copilot. It is used for writing, summarization of code, and emails, and much more. Now, the next
[18:18] emails, and much more. Now, the next part is natural language processing. It enables AI to understand, interpret, and generate human-type languages. It powers chatbots, voice assistants, and translation tools. Next, it helps
[18:33] improve communication between humans and AI. Now, let's proceed ahead and understand the everyday use cases of AI assistants. Starting with creative writing and content creation. You can create and
[18:47] generate blog posts, articles, social media captions, and scripts. AI assistant in brainstorming ideas, editing text, and improving clarity. And lastly, you can also create poetry, stories, or some song lyrics using AI.
[19:03] The second one is organizing and planning. You can make use of AI to plan a trip itinerary based on preferences and budget. You can generate meal plans and grocery lists as well. It can schedule events and set reminders and
[19:17] manage to-do lists. The next use case is personal assistance. You can use AI to draft and refine emails or messages. It also helps you by providing phone books and recommending movies and music. And
[19:31] lastly, summarize documents, articles, or even meeting notes. Lastly, we have some other practical uses which include the following. Learning new skills with AI-generated study plans, translating languages to
[19:44] provide real-time interpretations, offering health and fitness suggestions Now, proceeding ahead, we will understand AI agents. AI agents are advanced virtual assistants designed to perform tasks
[19:58] autonomously. They interact with users, process information, and execute commands. Some of the examples are ChatGPT, Microsoft Copilot, Gemini, and others. Now, what do AI agents actually do?
[20:13] So, AI agents conduct the following tasks. First, automate some repetitive tasks. Repetitive tasks like scheduling meetings, setting up reminders, and managing emails is done by AI assistants. It can enhance
[20:26] decision-making by providing data-driven insights and recommendations. Third is it can act as a personalized digital assistant which can help you with learning, travel planning, productivity, and much more. Lastly, integrating with
[20:40] tools and apps which can help you work across Microsoft 365, CRM platforms, and And now, let's proceed with the major part of today's session where we have to part of today's session where we have to focus on how to use AI responsibly and
[20:55] effectively. So, here we have a detailed Microsoft Copilot AI overview and also productivity. Let's go through that first. So, here we have a quick briefing of Microsoft 365 Copilot and we have also
[21:11] some links to the Copilot success kit, Copilot chat, agent starter kit, and the community links over here and resources, Copilot mobile version, what's new, and all and everything you need to know about Copilot, about the technical
[21:25] Now, back to the presentation where we were discussing about the responsible use of AI. So, the importance of ethical use of AI. AI should be used in a way that it is fair, transparent, and accountable.
[21:40] Understanding AI limitations will help you prevent misinformation and misuses. Ethical AI usage ensures trust and reliability in everyday applications. ensuring transparency and accountability. First one, clearly
[21:56] communicate when AI is used in content creation. Provide with the specific prompts that you're expecting in your output so that it cannot miss on the Secondly, we need to verify AI-generated information before sharing or acting on
[22:10] it because there might be instances where we might go wrong explaining our inputs or there might be some discrepancy in the data provided to analyze and generate a report. And lastly, understand how AI makes
[22:23] decisions to avoid over-reliance. Need to understand the thought process of an AI and expect what kind of results it might generate so that you don't have to rely on one single prompt every single time. Now, let's move ahead and address
[22:39] some potential biases in AI. AI can sometimes reflect biases present in its training data. Common biases include gender bias, racial bias, cultural bias, confirmation bias, and socioeconomic biases. Let's discuss each
[22:53] one of them. Firstly, gender bias. AI-generated job descriptions may reinforce stereotypes, example, associating engineers' roles with men. Second one is racial bias. Facial recognition systems
[23:07] may misidentify individuals from certain ethnic backgrounds. Third one, cultural bias. AI might prioritize Western perspectives in content generation. Fourth one, confirmation bias. It may reinforce existing beliefs by favoring
[23:23] information similar to past searches. Lastly, socioeconomic bias. For example, AI-generated loan approvals may favor individuals from higher-income Now, you also need to focus on responsible prompting. Use clear and
[23:39] neutral language to avoid influencing biased results. Ask AI to provide diverse perspectives and sources when generating responses. Be mindful of data privacy and avoid sharing sensitive personal information. Now, let's go
[23:54] ahead and check some of the prompt examples. So now, we are on the Microsoft Copilot prompt gallery, which will show you some of the sample prompts that you can make use of. For example, get a to-do list,
[24:08] add an image, quiz yourself, create a technology, write more confidently. For example, you might have a description, and you can just add it to improvise or add more impact onto it, find a specific information, draft an email, create a
[24:22] shopping list, and so on. So here, you can also go through some other prompts as well by clicking on view all prompts and a whole new world of prompt gallery will open up for you. Try it out and let us know your experience in the comment
[24:36] >> Are you feeling like every other video, article, or tweet you see these days is about generative AI? You're not wrong. It's the tech world's current obsession. But instead of just nodding along, wondering what it all means, what if you
[24:51] could actually master it? In this video, we're cutting through the hype, giving you a practical step you need to truly understand. I will be breaking down the entire journey into manageable phases, from fundamental knowledge to building
[25:05] your own Gen AI projects. Before we begin, let me tell you what generative AI or Gen AI means. As the name suggests, it is generating data. It's a technology where it uses existing data for creating new ones. It might be in
[25:20] the form of text, images, audio, video, code, or even 3D models. For better understanding, I will be dividing the entire road map into four phases, plus one additional phase as a bonus phase for you to excel in this field. Phase
[25:36] one is all about laying the foundation, which includes the basics of AI and machine learning. Think of artificial intelligence as making computers smart enough to do things which people can do. Machine learning is just one of the way
[25:49] to do it, allowing computers to learn from example instead of being told exactly what to do. It's a learning by seeing and doing it, not just by a rule book. There are three main types of machine learning: supervised learning,
[26:04] unsupervised learning, and reinforcement learning. Supervised learning is where the computers learn with correct answers, whereas unsupervised learning involves finding patterns on its own, and reinforcement learning is about
[26:16] learning through rewards. You will need Python too. Key tools include NumPy for numerical operations, Pandas for organizing data, and Matplotlib and Seaborn for data visualization. These help you to work with and understand
[26:31] data. You need to understand data preparation, which involves cleaning and organizing data. Also, learn about the feature engineering, which is the process of selecting important parts of the data of the model. Gen AI uses deep
[26:45] learning. You can think of deep learning as a computer network with many layers. Understand the basic idea of how these networks learn from data. This is a key for understanding more complex Gen AI models.
[26:57] Moving on to phase two, where you can start exploring the Gen AI models. Some of Gen AI models includes GANs, which are two networks competing, one making fake data and other tries to spot it. This model works well for creating
[27:13] realistic outputs. The second model is VAEs. This model learns a hidden representation of data to create new or similar data. The next model is transformers, which are excellent at understanding context and sequence like
[27:28] text, audio, and even images using attention. The last model is diffusion model, which can create high-quality data by reversing a noising process. This model is very popular for images. Apart from Gen AI models, you need to
[27:43] cover text generation model, which falls under natural language processing, which is NLP. These models power text creation from writing articles to translating languages. And lastly, you will cover image generation using GANs and
[27:57] diffusion models. In phase three, we will focus on developing our practical skills. You can start with small projects such as generating basic images or text with small data set. Do experiment and learn the workflow. Join
[28:11] platforms like Hugging Face, Reddit, Stack Overflow, and Kaggle forums to ask question and learn from others. To showcase your work, you can use GitHub to share your projects and code. Phase four is truly about staying ahead. Gen
[28:25] AI is always changing, so you need to be aware of it. New architectures are constantly being built, so briefly be aware that more advanced models are continuously being developed beyond the basics that we have covered. I would
[28:39] also recommend learning about deploying models on cloud platforms like AWS, Google Cloud, or Azure, which will be useful for larger projects. Additionally, you can read research papers, follow tech expert online,
[28:53] whether on Twitter or LinkedIn, and stay curious about this field. Continuous learning is the key in this field. Now that you have built a strong foundation in generative AI, phase five is all about mastering the future of
[29:06] technology. A certification course provides formal validation of specialized knowledge and skills, making individuals stand out to employees and demonstrating a commitment to professional development. This
[29:20] recognition often leads to enhanced credibility that improves career opportunities with specific field. Wondering which certification to go with? Don't worry, I've got you covered here as well. That's your road map to
[29:32] understanding generative AI. Start with the basics and get practical experience, and never stop learning. The power of Gen AI is within your reach. >> Hey everyone, welcome to Simply Learn's YouTube channel. In this tutorial, we
[29:46] YouTube channel. In this tutorial, we will learn about OpenAI's GenAI agents. So, generative AI agents are advanced AI-driven systems designed to autonomously perform tasks, generate content, and assist in decision-making
[29:59] by leveraging large language models, or also known as LLMs, and machine learning algorithms. Imagine an AI agent automating repetitive processes, enhancing productivity, and providing personalized recommendations across
[30:13] various domains, including customer support, marketing, and software development. By understanding context and generating human-like responses, they improve user engagement and streamline workflows. Businesses benefit
[30:27] from cost savings, faster problem solution, and improved efficiency. Such are a few out of many use cases of AI agents. Today, we will cover the OpenAI GenAI agents and their facts, their challenges, and future implications.
[30:43] videos you'd like to watch, then hit that like and subscribe buttons and the bell icon to get notified. Open AI Gen AI agents. So, first, let's get started
[30:55] will have a brief introduction to generative AI agents. Followed by that, we will understand the core components of Gen AI agents, then how generative AI agents work, architectures and deployment, enhancing AI agents with
[31:09] external tools, challenges and limitations of Gen AI agents, future trends and advancements in AI agents will be the last one. So, I hope I made get started with the first part, which is introduction to generative AI agents.
[31:24] So, basically, what exactly do you mean by the term generative AI agents? So, an AI agent is just an LLM that can take actions, and it can do so
[31:36] autonomously without any human supervision, and now also reason about the tasks. An AI agent is software entity that autonomously interact with environments, make decisions, and execute tasks. Generative AI agents
[31:51] execute tasks. Generative AI agents extend by leveraging large language models like Open AI's GPT to generate intelligent responses and perform tasks Now, we will also look into the evolution of
[32:06] AI agents. Traditional rule-based AI agents had limited to predefined rules. Like, if you define a process and give the steps, it will just perform them and complete it. It will not take a step beyond it. And machine learning-based AI
[32:22] agents can learn from data, but need a structured inputs. And then comes the generative AI agents, which are capable of understanding, reasoning, and generating human-like responses autonomously. Like, they understand the
[32:35] environment and learn through it and take their steps based on the principle will provide the reason for which they have taken a certain step. Now moving ahead, why generative AI agents matter a lot. So there are three key points to
[32:51] consider. Firstly, efficiency. They automate repetitive tasks with minimal human intervention. Next is high scalability. They handle large volumes of work, example customer support, content generation, and many more. And
[33:04] finally, intelligence. They can adapt to different use cases and dynamically. Nobody has to write rules for them. Now let's focus on OpenAI and its latest release of agents. So OpenAI just released two new groundbreaking agents,
[33:19] the operator agent and deep research agent. These two releases are absolutely massive for all AI agent developers. So operator is the first AI agent released operator is the first AI agent released by OpenAI. And what it does is it mimics
[33:34] human actions on your browser. So it means it can scroll, type, click, and navigate Google web pages for you just like any other human would. But more importantly, and what many people have missed out is this agent is actually
[33:50] based on a completely new model called CUA, which stands for computer using agent. So this agent, unlike standard GPT models, wasn't trained to just
[34:03] output text. You like write a prompt and expect an output in the text format. Instead of that, it can go beyond it. This agent was actually trained to output mouse and keyboard clicks, which is exactly what makes agent so powerful
[34:18] is exactly what makes agent so powerful in your browser. But not only that, it can also reason about every single action. This is also something that we haven't seen before. This agent doesn't just perform actions in your browser, it
[34:31] actually reasons about every single step and what it should do next. Just like we discussed before about the generic overview of generative AI agents, same as the Open AI operator here. It takes the step based on the environment, and
[34:46] also it will give you a particular reason for which or based on which it took a certain action, and it also asks you what to do next. Let's say if you want to book a ticket, it'll navigate to all the websites, it go through some uh
[35:01] better flight options for you, let's say the low-cost ones, the low uh hours of flight ones, and best timings, and then finally selects the best one for you based on your recommendations, and asks you if it should proceed with
[35:16] and asks you if it should proceed with booking or not. So, that's how it works. limitations as well. The first limitation is it has still a lower bit of accuracy. It's really high compared to other projects that we have seen in
[35:30] the past where people just prompt the GPT models to perform actions in the browser. It can browse web using GPT-4 or vision capabilities, but it often whenever there was a model on a web page, like some other random model
[35:45] available working readily on a web page, it would just accidentally pop up, and it would not be able to do anything because it would just get confused. It would not be able to close it, and then basically you would have to get
[35:59] and close this model manually. And the second limitation is high cost. Right now, this particular operator agent is only available on $200 plus plan. So,
[36:12] we can probably guess it's going to be pretty expensive in the API. Also, I do think that if you are focusing on the right thing, then costs are not going to Now, let's talk about the second agent that Open AI deployed. Now, the second
[36:26] agent which was deployed by Open AI is called as Deep Research. So, this second agent released by Open AI was trained to perform comprehensive research. It can search the web. It can pull up the necessary resources and then it can
[36:40] compile all that information into a really comprehensive document or maybe report as well. The groundbreaking thing about this agent is it's powered by the new O3 model. So, again, similar to the operator agent, but it doesn't just
[36:54] search the web and provide you with results. It actually reasons about every single source and then it thinks about what other information it needs to find out next and it's really incredibly powerful. Deep Research doesn't just
[37:07] internet just like some other standard GPT models. It would compile novel insights. Now, let's move on to the real-world applications. There are a wide variety of real-world applications. First one is customer support AI agents
[37:22] which helps in automating help desk responses. Next, powerful AI systems like co-pilots for professionals. Third one is research and analysis agents for data summarization, literature reviews, and many more.
[37:35] Fourth one is creative writing and content generation, automating social media posts, emails, etc. And lastly, AI-driven code assistants like GitHub co-pilot, Open AI codex, and many more. Now, let's go through the next part
[37:50] which is core components of these generative AI agents. So, the foundation models are GPT-4, DALL-E, and Whisper. GPT-4 for natural
[38:03] language processing, DALL-E for image generation, and Whisper for speech-to-text conversion. The next one would be prompt engineering and context So, here, designing effective prompts for AI agents to generate relevant
[38:18] responses, using few-shot and zero-shot learning for adaptability, and lastly, implementing context-aware AI agents that remember past interactions from the user. Now, next would be the memory and long-term context storage. AI agents
[38:35] typically process short-term prompts, but memory-based approaches, for example, LangChain's memory module, enable long-term reasoning as well. Now, storing and retrieving user-specific data for personal responses is also one
[38:49] of the priority. Next will be the tool use and API integrations. Using external API to fetch real-time data, for example, weather, stock prices, news, etc., and integrating databases for structured information retrieval. And
[39:03] lastly, further, we have autonomous decision-making. AI agents evaluate multiple options before making a decision. Planning and reasoning techniques to enhance responses is a priority for them. Proceeding ahead, we
[39:17] will discuss how generative AI agents work. Starting with the first step, input processing and context awareness. AI agents analyze user input, detect intent, and extract entities. For
[39:32] example, in a travel booking AI agent, it identifies destination, date, and budget from user queries, just like the example we discussed before, right? You provide the point A and point B, the best time for you to fly, and the best
[39:46] number of hours that you focus on flying, and lastly, the destination, right? All these things will be evaluated in real time, the cost, the number of hours, the flight, the options provided, the amenities
[40:00] things, and also budget as well. And based on that, they take a decision. Now, proceeding ahead, generating intelligent responses. So, here RAG, or also known as retrieval-augmented generation, AI AI external databases for
[40:16] responding. Next, we have chain of thoughts reasoning or also known as COT reasoning. And AI breaks down problems into fine steps. And last step is fine-tuning the models. Custom-trained AI for specific industries. For example,
[40:30] if you're preparing an AI agent for a dedicated sector like healthcare, legal, customer support, based on that, you will fine-tune your AI agent or model. Now, next is decision trees and multi-step reasoning. AI can simulate a
[40:46] thought process, breaking down complex questions into manageable steps. For questions into manageable steps. For example, whenever you open Deep Seek or whenever you open OpenAI ChatGPT for the most premium version, you will write a
[40:59] prompt and there will be an option called think, right? If you just click on that, it will explain you the thought process it is going through and how is it planning on giving a reason for you, right? And followed by that, we have
[41:13] handling uncertainty and error correction. AI self-corrects through loops. Last step in this particular stage is fine-tuning and reinforcement learning or also known as RLHF.
[41:28] Human-in-the-loop training to improve responses and AI learns from real-world interactions and adapts over time. So, it's basically called over time learning where it adapts to the real-world environment and learns things and
[41:40] provides the best feasible solution. Now, the next stage is architecture and Now, the next stage is architecture and deployment of AI agents. So, in the first one, we have standalone versus multi-agent systems.
[41:53] So, firstly, the standalone AI agents. They perform tasks independently. For They perform tasks independently. For example, ChatGPT. Multi-agent systems, in the other hand, work together to achieve a complex task. For example, one
[42:06] agent for summarization, another for execution. Now, next in this particular stage is agent orchestrated model. A central AI agent coordinates multiple AI sub agents. For example, in an AI powered
[42:21] research assistant, one agent gathers all the data that is required, the second agent summarizes all the information collected, and the third agent formats the output. Followed by that, we have cloud-based AI
[42:35] Followed by that, we have cloud-based AI agents and edge-based AI agents. Cloud-based AI agents compute. For example, OpenAI API, Azure AI. And edge-based AI agents run on local devices for faster processing. For
[42:49] example, Tesla's AI in self-driving cars. Followed by that, we have using APIs to build AI agents. OpenAI's API for GPT-4 powered by AI agents, LangChain framework for AI agent orchestration,
[43:05] and LlamaIndex for document-based AI agents. The next stage we will discuss about enhancing AI agents with external tools. In that, the first step is integrating with databases. AI agents can query SQL
[43:20] and non-SQL query databases for structured data retrieval. Proceeding ahead, we have connecting with web scrapers and APIs. AI agents can fetch real-time data using tools like Beautiful Soup, Scrapy, and OpenAI
[43:34] plugins. Next ahead, using vector databases for context recalling. FAISS, ChromaDB, store long-term memory for AI agents. For example, legal AI agents retrieve case law from vector
[43:50] embeddings. Lastly, we have autonomous agents and workflows. Auto GPT and Baby AGI are best examples. These AI agents that self-improve by learning from previous tasks. And Crew AI, which is a set of
[44:04] multiple AI agents collaborating to solve a complex problem. Now, the main part, challenges and limitations of generative AI agents. In that, the first one will be the ethical considerations. AI bias and
[44:18] fairness concerns are the primary ones. Addressing AI hallucinations, which is false information generation, and then we have computational cost and efficiency, which is a major concern amongst all the industries. Large AI
[44:31] models require significant computational power. Strategies to optimize AI agent efficiency will also considered to be expensive to hire many training models or also human programmers. Followed by that, we have security risks
[44:44] and data privacy. Risks of data leaks or data breaches and AI jailbreaking is one of the highest security threat. Implementing secure AI architecture is need to focus on. And lastly, we have legal and compliance
[44:59] issues. This would major be concerned from the enterprise level. GDPR, HIPAA, and AI regulations compliance is mandatory, and AI liability in mandatory, and AI liability in decision-making is also important. And
[45:14] discussion, which is future trends and advancements in AI agents. Starting with the first one, AI agents with emotional intelligence or also known as affective AI is capable of detecting emotions and adapting responses accordingly. For
[45:31] example, if you have designed a therapeutic AI chatbot for mental health. Now, the next one, self-improving AI systems. So, meta-learning and auto ML are a few examples. AI agents that learn
[45:44] from new data autonomously. For example, AI adjusting marketing campaigns based on customer responses. And now, the next one, AGI and roadmap to superintelligence. Well, this always ends up with question. Will AI agents
[46:01] evolve into AGI, or also known as artificial general intelligence? And are the implications of self-learning AI real? This is the major question. And lastly, the role of AI agents in the future of work. Now, we can see there
[46:15] are a lot of AI agents like Copilot, which will assist humans in their day-to-day task, whether it is business, health care, or education. And also, you have a lot of automation and transformation happening in the IT
[46:29] industry, which will reduce the work hours and stress on the employees by automating all those mundane tasks by the help of Copilots, also the generative AI agents. >> Welcome to the short course on
[46:42] generative AI models for beginners by Simply Learn. So, in this course, we will learn about the basic understanding of generative AI. Then, we will move on to the see applications of the many GenAI tools.
[46:58] And thirdly, we will understand the GenAI models, and then we will create our first GenAI model. And then, we will move on and see the real-world impact of the generative AI models.
[47:13] And then, we will create a chatbot using the GenAI model. And then, we will conclude the course with the limitations of the GenAI models, and finally, we of the GenAI models, and finally, we will see the future of GenAI models.
[47:27] Now, let us begin with the first chapter, that is basic understanding of generative AI. So, let us begin with what is generative So, generative AI is a type of artificial intelligence that creates new
[47:42] artificial intelligence that creates new content, such as text, images, music, or content, such as text, images, music, or even code from scratch based on what it has learned. So, unlike the traditional AI, which just analyzes or classifies
[47:56] data, like recognizing faces or recommending products, generative AI creates. It's like the difference between a judge and an artist. One evaluates and the other one imagines and produces.
[48:09] So, let's break it down with a simple real-world examples. The first one is real-world examples. The first one is ChatGPT, that is a text generation. So, what it does? So, ChatGPT, like the one you're talking to right now, generates
[48:23] you're talking to right now, generates human-like conversations and answers. So, you can ask it to write stories, emails, explain difficult topics, or emails, explain difficult topics, or even debug a code. For example, you say,
[48:36] "Write me a bedtime story about a robot and a unicorn." The ChatGPT creates an entire story for you on that particular spot. So, let's move on and see DALL-E, that is for the image generation.
[48:51] So, DALL-E creates pictures from words, that is the text prompts. Then, artists and marketers use it to Then, artists and marketers use it to generate logos, illustrations, or more
[49:03] without hiring a designer. For example, you type a cat wearing a For example, you type a cat wearing a spacesuit on Mars, and DALL-E draws it instantly. Now, moving on to GPT-3, that is text plus logic task. So, what it
[49:17] is text plus logic task. So, what it does? So, GPT-3 is a powerful language model that can create and write essays, summary books, translate languages, and even write computer code. So, developers use it in apps to create smart features,
[49:32] like the AI writing assistant. So, then you give GPT-3 a blog title and it writes the full article with an intro, subheadings, and conclusion. So, after knowing all this, let's move on and see why are generative models. So,
[49:47] generative models are changing the way we create content across multiple fields. So, under the text field, we are writing books, articles, reports, or even poetry.
[50:00] And then we are assisting in the customer support with the chatbots. And then under the images section, we are helping the artists generate concepts and visuals. And then we are even handling the
[50:14] speeding up design and marketing content creation. And then under the music part, we are composing the background scores for videos or games. And then we are helping musicians brainstorm the melodies or the lyrics.
[50:30] And then finally, we write the boilerplate code under the coding section, debugging, or even creating full web apps. So, now moving on, we'll full web apps. So, now moving on, we'll see traditional AI versus generative AI.
[50:44] So, let's compare both side by side with a simple explanations. So, the main goal of traditional AI is to analyze and make decisions. And the main goal of generative AI is create new content.
[51:00] For example, under traditional AI, predicting stock prices or recommending books on movies. And in the generative AI, we are writing stories, generating images, and so on. So, traditional AI handles data, it uses data to classify
[51:16] or forecast, and generative AI learns from data to generate something new. The traditional AI, it labels, scores, and categories as output, and generative AI, the text, images, music, videos, and so on. For example, traditional AI is face
[51:33] recognition, fraud detection, spam filters, and generative AI is ChatGPT, DALL-E, MusicLM, and so on. So, down the line in the next chapter, we will see
[51:45] the applications of some generative AI tools like the ChatGPT, DALL-E, and GPT-3. So, imagine this. You are working on a college project. It's late at night, and the deadlines is tomorrow. You need a well-written report and a
[52:00] poster design and a clear presentation. So, total chaos, right? But what if I tell you that there's an AI assistant that can help you with all of this in just minutes. So, welcome to the world of generative AI. Today, we'll explore
[52:14] how tools like ChatGPT, DALL-E, and GPT-3 are being used by the students, content creators, and professionals to do amazing things faster, easier, and smarter. So, let's break down each tool and see how you can actually use them in
[52:30] and see how you can actually use them in real life. So, this is ChatGPT-4o. If you're not aware of what ChatGPT is, so ChatGPT is an AI chatbot that can create, write essays, summaries, articles, explain topics, solve coding
[52:45] problems, and even conduct mock interviews all through one single chat. it. So, ChatGPT model, it uses this GPT-4,
[52:58] the latest in the generative AI pre-trained transformer that is a GPT pre-trained transformer that is a GPT family. So, GPT-4 is designed for more nuanced and human-like conversation. So, have a look at the real ways we can use
[53:12] it. So, the first prompt we are going to try is explain the greenhouse
[53:25] like I am in class 12. the greenhouse effect explanation, and
[53:39] it is asking the ChatGPT to behave that it is explaining to a class 12 person. it is explaining to a class 12 person. So, we will give this.
[53:53] generation. Just like here the greenhouse effect is a natural process that warms the earth's surface. It happens when certain gases in the earth's atmosphere trap the heat. Here's how it works. It will have a detailed
[54:07] explanation on the solar energy earth surface absorption, greenhouse gases, heat trapping, and all. So, after all these four points, it has also given without the greenhouse effect, the earth would be too cold to sustain life.
[54:22] However, human activities like as burning fossil fuels, deforestation, and industrial processes. So, you can basically think it has given one nutshell. And if you just understand this, you have a clear vision of what a
[54:36] greenhouse effect is. So, the first point we can see over here is that it is like a own tutor. So, the first point we have just been to Chat GPT that is founded by chat.openai.com. We have typed a question and we have got
[54:52] a simple and easy to understand answer. And even you can have a follow-up And even you can have a follow-up question. Like if I ask, is solar energy related to
[55:11] effect? You can even have a follow-up question. Yes, solar energy is related to the greenhouse effect. So, it's almost like it's own tutor. So, great for the late-night cramming or
[55:23] the quick concept clarity. So, this is it. And now moving on to the second thing that it can help with, that writing assignments or essays in just
[55:35] minutes. So, we will ask Chat GPT, write a 350 word essay on
[55:47] climate change for for college purpose. conditions. You can even customize it by saying make it simple and yet
[56:04] and awesome. So, it will basically create it has even created a one title, a growing concern Climate change it has created paragraphs and basically you can even customize
[56:21] this word count and topic then review the generated draft. You can even the generated draft. You can even personalize it to apply this personality and done and dusted. You can just have this. Now writing feels less stressful
[56:34] and it would be more fun. It can even give you an interview experience. For example, just imagine you are having your interview on the next Monday and you are preparing and you are very confused about what to prepare and what
[56:48] not. Then you can basically ask Chat GPT to behave like a interviewer for a particular job. I can even paste your job description from that particular company and you can ask that what are
[57:01] the probable questions I can have. So, let's have a demo. Ask me three HR questions
[57:16] for a software internship and suggest
[57:28] and suggest improvements. And in no time it has started generating. Here are the three HR internship. The first one, can you walk me through a recent coding project you
[57:44] have worked on and explain how you approach solving the problem? And you know what? This is a very very common question and it has given one tip also, suggestion. This question evaluates your problem-solving skills, your technical
[57:58] ability and how you approach development task. It gives the interviewer insight into your practical experience and thought process. So, how do you manage multiple projects or the task simultaneously? So, this question
[58:14] assesses your time management skills and ability to prioritize the task. The software development often involves juggling multiple assignments. So, this gives the interviewer an idea of your organization skills. So, this is very
[58:28] important. It has also written, how do you keep yourself updated with the latest programming languages and technologies? So, basically this technologies? So, basically this question looks after your own learning
[58:40] So, the self-improvement part. So, it has even created one and answer over it. And it has also suggestions for the improvement, provide specific examples, highlight the soft skills, prepare for the problem-solving questions and so on.
[58:56] And if I for example, if I write for example, if I write customize it with a data analyst
[59:11] fresher role having having no experience.
[59:33] And it will give you answer in no it will give you answer in no time. career as a data analyst? And what are the steps you have taken to prepare
[59:47] yourself for this role? Can you describe a situation where you had to analyze data or solve a problem using numbers or data in your studies or personal projects? How do you approach learning new tools or technologies, especially
[1:00:00] when it comes to data analysis software? And then we have this technical question. What is the difference between the structured and unstructured data? What is data normalization? How comfortable are you with the data
[1:00:12] visualization tools? Can you describe a visualization you would use to represent sales data over time? Can you explain what a pivot table is and where it is used in data analysis? And there are suggestions that you are showcase your
[1:00:26] learning journey, focus on the problem-solving skills, prepare for the basic technical questions. Since you are a fresher, the technical questions may focus on the fundamentals like the Excel, Python,
[1:00:38] that is the libraries, Pandas, and so on. So, in no time it can even have a mock interview over here. So, you can even create ideas over here. It can brainstorm you with the ideas available.
[1:00:53] For example, you have a group project and you want the idea. I'll type over here, give five creative ideas for a
[1:01:23] in no time it has created these five ideas. Sustainable packaging design challenge. Objectives design a sustainable It has created one objective. It has given one approach for you and one outcome. Then
[1:01:38] it has created community-based urban garden. And for each topic, if you see, for each idea, if you see, it has created the objective, the approach, and So, these projects can not only be educational, but also impactful. So, you
[1:01:54] never run out of inspiration from ChatGPT. So, ChatGPT is a lot. And here, ChatGPT. So, ChatGPT is a lot. And here, if you can actually see, it has various search, deep search, and create image. And you can basically do everything you
[1:02:08] want from ChatGPT. Let's now move on to DALL-E. The next we would be seeing is DALL-E. So, DALL-E is an AI that creates images from text. You tell it what to draw, and it generates the artwork,
[1:02:22] logos, and visuals. So, you would be getting DALL-E from this ChatGPT only. And in the explore GPTs, in the sidebar, you have to just mention DALL-E
[1:02:36] 2. So, there are various versions of DALL-E. And we'll start chatting. And now, if you close this sidebar, you can basically see
[1:02:50] DALL-E 2. So, DALL-E is an AI that creates the images. And DALL-E, this model, uses a version of the CLIP model. That is a contrastive language image That is a contrastive language image pre-training. Combined with the VQ-VAE 2
[1:03:04] or other generative image models. So, DALL-E creates images from the text descriptions, generating highly creative visuals based on the natural language visuals based on the natural language input. So, let's see how you can use it.
[1:03:17] for example, we you're going to design one of the college poster design. So, instantly it would be done. we would write create a poster
[1:03:32] for a college fest DJs, {comma} neon lights,
[1:03:45] and a
[1:03:58] And we can basically give the prompt. And in no time, it will create one great image for you. You can write the And you can see it has You can write the And you can see it has created a great a great image.
[1:04:11] You can basically download this image and do anything. featuring a vibrant DJ scene with the neon lights and the energetic crowd.
[1:04:23] So, forget Photoshop. Let AI do the heavy lifting. So, then you can create presentations over here adding better visuals. over here adding better visuals. So, I'll write draw a cartoon
[1:04:37] So, I'll write draw a cartoon showing
[1:04:52] in a river. And Now, you can insert it into the PowerPoint or Canva and design it on
[1:05:04] your your own. It acts as a vector, it acts as icon, and anything you want to Then you can instantly create club logos and branding. So, you can even generate a
[1:05:18] logo for over here through DALL-E 2. Logo for a coffee club
[1:05:30] books and steaming and steaming mugs.
[1:05:42] logo and it feels like real. So, you have just given one prompt and in and get four to five variations of logo. You can even ask it for a different version. So, great for the student clubs or college
[1:05:56] Then, you can even have it for the costume design or any ideas again. So, we'll try and futuristic
[1:06:10] design for a for a sci-
[1:06:22] with glowing elements.
[1:06:35] it has created a futuristic costume design for a scientific fiction play with the glowing elements. And it is very amazing. The qualities of the very amazing. The qualities of the images are of high quality.
[1:06:52] detailed prompts and you will get a great great output from DALL-E 2. art students and theater clubs, this is your new secret weapon. So, let's move on to GPT-3. Our next GenAI tool is GPT-3. So, GPT-3 is the
[1:07:10] powerhouse engine behind the tools like ChatGPT. So, it doesn't just chat, it can write full code, research summary papers, business emails, and more. So, papers, business emails, and more. So, GPT-3 is a predecessor of GPT-4 based on
[1:07:25] the same GPT architecture, but with a 175 billion parameters. It was one of the most advanced language models when it was released. So, let's explore how it helps in the real-world tasks. So,
[1:07:39] initially, we can use ChatGPT because it is a ChatGPT 4 version. There is a GPT-3 is basically a predecessor of GPT-4. So, I'm using ChatGPT over here. You can also use ChatGPT 4 to understand the GPT-3. So, the first one is building
[1:07:55] websites without coding. So, you can even write write a HTML for personal
[1:08:11] portfolio with my name, and contact form.
[1:08:28] You can just copy it, paste it in VS Code, or any of the code editors you proper output of this.
[1:08:42] Then, you can even summarize a lengthy research paper. You can have to just upload the paper, and if you see this plus section, you can just
[1:08:54] click on this, you can just upload from your computer, and summarize it in 2 seconds. It can even brainstorm you with the project and the startup ideas like suggest
[1:09:12] app ideas for for college students.
[1:09:24] Now, you can see the first one is study buddy finder. Help students find partners based on their courses, schedules, and many preferences. And you can even see campus event hub, course material exchange, mental wellness
[1:09:37] material exchange, mental wellness check-in, foodie finder, and so on. It has even created bullet points for a better understanding. So, it is a noise-free study timer. Use it to kickstart your next hackathon or
[1:09:51] competition or anything. So, I'm not dragging it more because as I have already showed you the use of ChatGPT. So, that's it. Moving on to the next lesson of understanding the generative AI models.
[1:10:06] generative AI models. Imagine you are teaching a computer to be creative, like painting a picture, writing a poetry, or designing a dress. That's exactly what generative AI models aim to do.
[1:10:21] So, learn from the data and generate new content that looks, sounds, or feels content that looks, sounds, or feels real. But, how do this pull off? Let's break it down into the three major families of models. The first one is
[1:10:34] families of models. The first one is GANs, that is the generative adversarial networks. Then the second one we have is VAEs, that is the variational autoencoders. And the third one, the final one is
[1:10:47] transformers, like the GPT. So, let's understand this in a detailed one. The first one is generative adversarial networks, GANs. The artist versus the critic game.
[1:11:01] So, how it works. Think of GANs like a competition between the two neural networks. The generator is like a rookie artist trying to create a fake art. The discriminator is the tough critic trying to tell the real art
[1:11:15] from the fake. So, the artist creates random images. So, the artist creates random images. The critic says, "Fake." or real, and the artist learns to get better. So, over the time, the fake images become
[1:11:28] indistinguishable from the real ones. So, there's a real-world analogy. It's like a student that is a generator forging a a student that is a generator forging a signature and a teacher discriminator
[1:11:42] spotting it. As both improve, the forgeries become nearly perfect. So, there are certain applications. The first application is creating realistic human faces, then style the transfer in the art, and then we have
[1:11:58] fashion design prototypes, and then finally, we have deep fakes that is controversial but technically fascinating. challenges. So,
[1:12:12] instability. Then, the second one is it requires a lot of data and computing. So, moving on, we'll understand variational autoencoders, that is the VAEs. The data compressor with
[1:12:27] creativity. So, VAEs are like the compress and the rebuild systems. They take input data like an image, compress it into a low-dimensional space that is called a latent space, and then
[1:12:42] called a latent space, and then reconstruct it. But, here's the twist. The VAEs don't just remember, they imagine. So, by sampling different points in the latent space, they can generate new similar looking content. So
[1:12:58] let's move on and see the applications. First application is anomaly detection. So what doesn't fit in the normal data? Then the second one is generating
[1:13:10] handwritten digits and faces. And then we have data compression. And then finally we can see drug discovery, that is generating molecular structures. So now let's move on to the strengths. So more stable than the GANs, and it
[1:13:27] gives a well-structured latent space for the interpolation. So now moving on to the third model, we have transformer models, that is a master of the So unlike the GANs or the VAEs, which focus
[1:13:42] more on images, transformers shine in handling sequences, specially the text. The core data idea is attention mechanism, a way to let the model focus on the different words when generating a sentence. So for example, when GPT
[1:13:59] generates a sentence I like ice cream because it's sweet, it pays attention to the context, understanding how because relates to the ice cream and sweet. So let's now move on to the applications. The first one is chatbots,
[1:14:15] like the ChatGPT. And then we have code generation, that is the GitHub Copilot. And then thirdly we have language translation, and finally we have summarization and content creation. And then let's move on and see why
[1:14:32] transformers stand out. So the first one is they scale data well. The second one is they can be fine-tuned on specific tasks, like the medical summarization or legal drafting. Then finally we have the handle, the long-range dependencies
[1:14:49] better than the older models like the RNNs or the LSTMs. Then, we have a comparison and the use cases. So, the first one is GANs, and it is
[1:15:01] best at image generation. And then, we have an example use, that is the face synthesis. And the associated strength is high realism, and the weakness is it is hard to train. Then, we are seeing the VAEs, and it is
[1:15:17] best at data compression, plus it is also good in anomaly detection. So, it is used as a latent space exploration, and then coming to the strength, it is very stable training, and then coming to weakness, it has a
[1:15:33] less sharp outputs. So, then coming to the model type, that is the transformers, and it is best at text and language. And then, we have an example use, that is a chat GPT and summarization. And then, we have
[1:15:48] strength, that is powerful with a large data. And then, we have certain weakness, that is expensive to train. Now, moving on, let's clear something. Which to use and when? So, if you don't
[1:16:01] know which generative model to use for what, it can mess things up fast. So, let's say you are building an app that needs to write emails, make pictures, answer questions, and turn speech into text. If you use the wrong model for the
[1:16:16] wrong job, your results will be poor, things will be slow, and you will waste time. So, let's understand which models performs what. So, for a sharp realistic images, we are using GANs. And then,
[1:16:31] coming to the next, for variation and latent space analysis, we are using VAEs. And then, finally, for the language or the text-based tasks, we are using transformers. So, finally, how do these models
[1:16:45] actually work? So, the GAN training process is the first process is generator makes the fake data. Then, the step two is the discriminator tries to classify the real versus fake. Then, the third step is both get better at their
[1:17:02] jobs with each round. So, the final goal is to fool the discriminator with the realistic outputs. So, the transformers under the hood pays attention mechanism. So, it helps model understand which words matter more. Then, we have
[1:17:18] self-attention that is allows the model to look at all the words in a sentence at once. And then, we have positional that is encoding and it helps track the orders of the words as transformers don't read left to right like humans.
[1:17:34] So, this is why models like GPT-4 can summarize, translate, and generate and text so well. So, understanding the differences between these generative models is like knowing which tool to use for which creative
[1:17:48] job. So, GANs are your digital painters, VAEs are the smart compressors with imagination, and transformers are your text wizards. So, down the line, we'll how to create our first generative AI model.
[1:18:05] In this chapter, we will be creating our first generative AI model. So, the objective of this lesson is breaking down the steps of building a generative AI model using a pre-trained model. So, we are here not training a
[1:18:20] model from the scratch. So, instead, we will fine-tune or directly use a pre-trained model. So, this makes it fast and beginner-friendly. So, if I tell the steps in a nutshell, it will be we will be first choosing a
[1:18:35] pre-trained model. For example, GPT-2 because it is lightweight and good for beginners. The source from which we will getting the model is hugging faces model hub, and then we will be setting up the
[1:18:48] see. Then we will be installing the libraries, the transformers, torch, and then loading the pre-trained model and the tokenizer. Then we will be writing a
[1:19:00] prompt and generating a text. So, here we will be using the pipeline API for simplicity and parameters like max length and temperature.
[1:19:12] So, let's begin. So, here I am inside this Google Colab and I have chosen a new notebook to begin with, and then I'll just insert pip install
[1:19:36] And shift enter. And may It may take some time. installing all the required transformers. pip install means install this package in Colab, and transformers is a library
[1:19:52] by the hugging face. So, it gives us access to the powerful AI models like GPT-2, BERT, and more. And then we will see
[1:20:04] And then we will see we will install from transformers pipeline. And then
[1:20:19] we will be using the loading generator GPT-2 text generator. So, generator and tap.
[1:20:32] and tap. So, And basically, this pipeline is a shortcut tool. It hides the all the complex code needed to run the AI models. So, here the text
[1:20:47] generation tells like, "Hey, I want a model that can write text for me." And GPT-2 is the actual model name. It was trained by the OpenAI to predict what word comes next in a sentence. And then basically,
[1:21:02] you can see that it has model 7 tensors, 100% has loaded everything. And and and the storage is also enabled for this repository. And then we'll begin with the prompt, okay?
[1:21:18] So, let's give the prompt. And let's keep it very simple. Prompt equal to once upon a time once upon a time in a world
[1:21:41] And then in the next line, I will be in the next line, I will be writing result is equal to generator.
[1:21:54] inside it. And then we'll be giving the max length one parameter. And I will be specifying it as 100. And I will be giving the number
[1:22:09] will be giving the number return sequence.
[1:22:23] And I don't really require this. So, I will be directly printing the result. I will be directly printing the result. The first one, the first doc will print
[1:22:39] we'll be passing the generated the generated text.
[1:22:54] a starting sentence you give to the AI. Like saying, "Hey, start a story with this and this and this." So, max length is equal to 100. So, this particularly means generate up to 100 words or characters after the prompt.
[1:23:09] And then you can even see I have given this num return sequence is equal to one. This is asking for just one version of the result. So, you can basically ask for two, three versions. I have just asked for only one. And then the result,
[1:23:23] And in the third bracket I have given a zero and the generated text. It extracts the generated text from the result. So, you will see the AI write a continuation of your prompt basically. So, that's your first generative AI output. I can
[1:23:39] basically see, "Once upon a time in the world of AI, your only limit was when you could kill monsters, but now you can for the first time ever." The new monsters are in the battle in the blood of elves now. Not the usual black and
[1:23:53] white battles that we You have experienced with the past couple of experienced with the past couple of years. And it has created a great one. So, you can even see the power of generative AI. It has the power of
[1:24:06] completing your text, your prompt. So, then let's write result equal to generator.
[1:24:35] enter then max length I will again mention then enter I want the num
[1:24:47] num return
[1:25:04] and have to put a comma over here, okay? So, don't forget this, guys.
[1:25:19] top key. Don't worry, I'll explain one by key. Don't worry, I'll explain one by one as I write.
[1:25:39] equal to 0.95 comma then I'll write do samples
[1:25:54] equal to true. Basically, a Boolean value. So, then I'll after this I'll print the result. Print result
[1:26:06] and in third brackets with the first one and then again third brackets generated text
[1:26:19] the basically going to shift enter. we have parameters like temperature. So, temperature, what it does, it controls
[1:26:32] the creativity. This is the lower is equal to safe, and higher is the wild, equal to safe, and higher is the wild, and creative is 0.7 medium. So, this top k top {underscore} k is equal
[1:26:45] to 50. So, pick from the top 50 likely next words instead of just the top one. Then, we have this top {underscore} p is equal to zero. It is the nuclear sampling. That is a more natural randomness. Then, we have do sample is
[1:27:01] equal to true. That is enables the randomness. So, without this, the output is too predictable. So, once you run this code, you will see something like once upon a time in a world full of AI, and with every
[1:27:14] advancement in AI machines, there has been a change in the way we think about the world. And we need to make sure that we can make sure that we can make sure and it has repeated it so many times. So, every time you run it, you might get
[1:27:30] a slightly different result. So, over here, you have used the hugging faces here, you have used the hugging faces transformers library to load the GPT-2. Then, you wrote a prompt, and then the AI continued it using its training. So,
[1:27:44] you called it. So, you did it and without any complex machine learning setup inside the Google Colab. So, this is your very first generative AI model. In the next chapter, we'll be seeing the real world of impact of
[1:27:59] generative AI models. So, generative AI is not just about another buzzword. It's fundamentally changing how businesses operate, solve problems, and serve customers. From generating product descriptions to
[1:28:14] designing aircraft parts, generating AI is stepping into roles that once required human creativity and intelligence. So, in this session, we will explore how industries across the board are using
[1:28:27] generative AI in meaningful and practical ways. The first one is healthcare. That is a faster diagnosis and drug discovery. So, in healthcare, generative AI is helping doctors and scientists make faster decisions.
[1:28:43] scientists make faster decisions. The first one is medical imaging. So, AI can analyze medical images like the X-rays or MRIs and create reports to help doctors quickly identify the problems. For example, a hospital uses
[1:28:58] problems. For example, a hospital uses AI to write reports for doctors, cutting down the time spent on each patient. Then, we have drug discovery. So, AI helps scientists discover new medicines faster by predicting how proteins fold,
[1:29:15] which is important for the creating drugs. For example, a company used AI to drugs. For example, a company used AI to discover a new drug in just 18 months, discover a new drug in just 18 months, which usually takes much longer time.
[1:29:28] Then, moving on to the next one, we have finance. That is a smarter risk management and report generation. So, in finance, AI is making it easier to analyze data and detect the problems. The first one is automated financial
[1:29:45] reports. So, AI can create financial reports based on data, saving time for analysts. For example, a hedge fund uses AI to write the market reports in
[1:29:57] Then, we have fraud detection. So, AI helps banks detect fraud by generating the fake transaction data to test their security systems, making them more reliable. So, for instance, MasterCard uses AI to create realistic transaction
[1:30:14] data to improve their fraud detection system. So, now moving on to the third we have retail and that is the personalized shopping experience. So, retailers use AI to create personalized shopping
[1:30:29] experiences for customers. The first one we have is a product descriptions. So, AI writes product descriptions that are customized for different customer groups
[1:30:41] such as casual descriptions for young buyers and formal ones for the professionals. For example, a clothing brand uses AI to automatically write descriptions for thousands of products. And then we have virtual try-ons. So, AI
[1:30:57] lets customers try on clothes or makeup virtually to see how it would look on them before buying. So, for instance, Sephora's app uses AI to suggest makeup
[1:31:09] and lets users see how it looks on their face using their phone camera. So, now moving on to the fourth one we have manufacturing. This is a faster have manufacturing. This is a faster design and maintenance. So, AI is also
[1:31:23] making manufacturing more efficient. The first one is AI generated designs. AI helps create lighter, stronger parts for products reducing waste and material
[1:31:36] products reducing waste and material cost. For example, an airplane company used AI to design parts that were 50% lighter but just as strong as a traditional designs. Then moving on we have predictive
[1:31:49] maintenance. So, AI helps factories predict when machines will need maintenance reducing downtime and improving efficiency. For example, a factory uses AI to automatically create maintenance schedules for machines based
[1:32:05] on their performance data. So now moving on to the fifth one we have education. That is a personalized learning for students. So AI is making education more personalized and accessible.
[1:32:19] The first one is AI tutors. So AI powered tutors can help students with the homework by guiding them step-by-step through problems. For example, a student stuck on a math problem can get hints and explanations
[1:32:34] problem can get hints and explanations from an AI tutor available at any time. Then we have automated feedback. So teachers use AI to create quizzes and provide feedback on assignments saving them hours each week. For instance, a
[1:32:49] teacher uses AI to review essays and provide comments in just minutes instead of hours. So now moving on we have sixth one is entertainment that is creating at
[1:33:01] a scale. So in entertainment AI is being used to create content faster and cheaper. First one is script writing. AI can help writers come up with ideas, generate dialogues, and even create storyboards before filming. For example,
[1:33:17] a filmmaker uses AI to plan scenes and create visuals before they start shooting save time in the pre-production. So then we have virtual characters. So AI can create virtual characters that speak and act which is
[1:33:32] useful for movies, games, and social media content. For instance, AI is used to create virtual influencers who interact with their followers on social So now moving on down the line we will see how to create a chatbot in Google
[1:33:49] see how to create a chatbot in Google Colab. So before we dive into the code, let's first understand what the hugging face transformers library is. So, the hugging face Transformers library is one of the most popular libraries for the
[1:34:03] natural language processing, and it provides the pre-trained models that can perform a variety of tasks like the text generation, text classification, named entity recognition, and more. So, here in the code, we will be using a
[1:34:19] model called DialoGPT-Small, which is a variant of GPT-2 designed for conversation generation. So, the DialoGPT was trained on the large conversational datasets, making it a perfect fit for building the chatbots.
[1:34:33] So, let's now build the setting up the environment. So, for this, we will environment. So, for this, we will install pip install install pip install transformers.
[1:34:51] transformers library. You need this to work with the pre-trained models provided by the hugging face. So, this will allow you to load the models like the DialoGPT we will be using here, a conversational AI model,
[1:35:05] responses. And then, and here we will begin with importing the necessary libraries. From transformers, import the auto model for the casual LM auto tokenizer, import torch, and then we will run this. So,
[1:35:22] here we are importing the necessary components like the auto model ID for the casual LM. So, this class basically loads a pre-trained model for the casual language modeling, which is used for the text generation task.
[1:35:37] So, we will use it to load the DialoGPT model. And the auto tokenizer, the tokenizer converts the text into tokens, tokens which are known as numbers. Now, the the can process and then converts the
[1:35:53] model's output back into the human-readable text. So, the torch is then PyTorch is a deep learning framework used by the model. It's used for the tensor manipulations as models rely on these tensors for the
[1:36:09] input and output. So, let's begin. Then we will begin with a loading the model and the tokenizer. So, here this converts the raw text that is the user input into tokens.
[1:36:29] word or phrase is mapped into the unique ID making it easier for the model to process and generate a response. So, then model is that pre-trained dialo GPT small model is loaded over here. So, this model has been trained on a large
[1:36:45] corpus of dialogues, which enables it to generate the relevant responses in a conversational manner. So, then we will begin with the chat initialization. And before beginning with the chat initialization, we will see that the
[1:37:00] tokenizer config.json is 100% complete. And the best part of Colab is it will give you all the status of how it is complete and for which particular merges.txt or vocab.json which is completing and
[1:37:15] what problem is there. So, let's just run this. let me explain. So, chat history IDs. So, this is a variable where we store the history of the conversation. So, this helps the
[1:37:30] model maintain the context across multiple user inputs. Then we have we're having a print statement to start chatting with your AI. Type quit to exit. So, this provides a welcoming message to the user to start chatting
[1:37:46] and gives the exit command. So then we will begin with a chat loop that is the main logic of the chatbot. So then we'll begin with this chat loop logic. So we have written for step in
[1:38:00] range five, user input is equal to input and then you and then semicolon and then a space. And then we have written if user input.lower is equal to equal to quit then I have to break it.
[1:38:15] Break the loop. Break. So then I'll run this. And this section of the code basically uses input to get the user's message and exit condition. If the user types quit, the loop will
[1:38:30] break and the chat ends. So let's see. And here basically you can see I have written here and still it doesn't break. So let's write quit over here and let's see what happens. So you can see that it has stopped generating.
[1:38:44] Means it has got out of the logic. So here we have this encoding user input. So encoding the user input is converted into token IDs using the
[1:38:56] tokenizer.dot.encode. So the EOS token that is the end of sequence token is added at the end of the user input. This tells the model that the input has ended and it prepares for generating a response. Then tensors.
[1:39:12] So this is a return_tensors equal to PT. That part ensures that the equal to PT. That part ensures that the encoded input is returned as a PyTorch tensor. This is the format that the model expects. So we will run this.
[1:39:26] And then we will be moving on to the managing the conversation in history. So here we are having this managing conversation history and it is up to maintaining the context. So we want to keep the track of the the
[1:39:40] history so that the model can respond in context. So, if there's previous chat history, that is a chat history IDs, it is concatenated with the new input, allowing the model to consider the
[1:39:53] entire conversation so far. So, if there is no previous history, that is the is no previous history, that is the first user input, the new input IDs are used directly. So, then we will be moving on to
[1:40:06] generating a response. So, here we are concentrating on a response. Chat history IDs is equal to model generated bought input IDs max length equal to 1,000 and bought token
[1:40:20] length equal to 1,000 and bought token IDs equal to tokenizer.eos_token_id. the concatenated input to the model, that is model.generate, and generates a
[1:40:32] response. The maximum length specifies the maximum length of the generated text. So, in this case, this set to 1,000 tokens. Then we have bought token ID, that is it ensures the padding tokens are handled
[1:40:48] correctly, that is the model needs to know which tokens are just padded. Then we will be moving on to the decoding the response. And here we are seeing response is equal to tokenizer.decode chat history IDs, that
[1:41:04] is the bought input IDs.shape and everything. So, basically decoding this generated token IDs are converted back into the human readable text using the tokenizer.decode. Then we have skip the special tokens.
[1:41:20] The skip special tokens is equal to true, it ensures that any special tokens, like the EOS, are not included in the final output. in the final output. So, we will run this.
[1:41:37] printing the response. That is displaying the response. So, the print this generating the response is displayed to the user. So, we'll run this. And you can see, "Bot, I'm so sorry."
[1:41:51] So, what happens when you run this code? You initiate the chat, and the bot welcomes you to start chatting. So, every time you input something, the bot takes your message and codes it and sends it to the model along with the
[1:42:05] generates a response considering the entire conversation so far. And the bot outputs that response. So, this loop continues until you type quit, ending the
[1:42:19] conversation. So, building a chatbot is essential because it enhances the user engagement by providing the real-time personalized interactions, making it more dynamic and the user-friendly compared to the static
[1:42:32] content. So, chatbots are available 24/7, ensuring the users can get immediate responses at any time. So, by automating the repetitive tasks like answering common questions, processing orders, or providing basic customer
[1:42:47] support, chatbots free up valuable resources, allowing human employees to resources, allowing human employees to focus on more complex tasks. So, then, additionally, chatbots can improve customer satisfaction by quickly
[1:43:00] resolving the inquiries, reducing the wait times, and enhancing the overall user experience. So, implementing the context management allows the bot to remember the previous interactions, ensuring a continuous and more
[1:43:16] personalized experience for the user. So, finally, enabling the multi-turn conversation handling means the chatbot can engage in back-and-forth dialogues rather than only answering the isolated questions, so making it feel more
[1:43:31] conversational and human-like. So these features collectively help create an intelligent, efficient, and engaging chatbot that can significantly improve user interactions and streamline the business operations.
[1:43:45] So you can even fine-tune your own model. So this could be your own data set to make it more specific to a particular domain or a use case. You can even make it worth handling the
[1:43:59] longer conversations so that the it remembers the chat history for a long time. So guys, you have made your first chatbot and down the line, let's get improved.
[1:44:12] in this lesson, we will understand about the limitations of generative AI models. So the first one is lack of understanding. So AI models generate contained by predicting what comes next based on the
[1:44:28] patterns in data. But it don't truly understand the content. For example, when an AI creates an article or a code, it doesn't know what it's writing. It's just mixing patterns based on the past data.
[1:44:44] data quality. So the AI's performance is only as good as the data it's trained on. So if it has biased or incomplete data,
[1:44:56] it can generate biased or inaccurate results. Think about biased hiring algorithms or flawed health care recommendations. Real flawed health care recommendations. Real issues caused by the bad training data.
[1:45:09] So now moving on to the third one we have creativity limitations. have creativity limitations. So AI can produce impressive creative outputs like artwork or music. So however, it struggles to innovate or
[1:45:23] So however, it struggles to innovate or produce truly original ideas like a human artist would. It's more about remixing the existing content rather remixing the existing content rather than inventing something groundbreaking.
[1:45:35] Now moving on to the next, we have ethical concerns. So AI models can sometimes generate harmful content. From offensive language to deep fakes
[1:45:47] that are hard to distinguish from the real footage. real footage. So this poses risk for misinformation and ethical dilemmas, especially in media and entertainment.
[1:46:01] So now moving on to the next, we have context and nuance. So generative AI can miss the subtle nuance in conversations or emotions. For instance, if you ask a model to help with the sensitive topics
[1:46:16] like mental health, it might offer advice that's generic and more tailored advice that's generic and more tailored to the individual's emotional needs. Next we have cost and resource intensive.
[1:46:29] So training a generative AI model requires huge amounts of computing requires huge amounts of computing power, time, and energy. So these models often run on the powerful servers consuming a lot of resources, which
[1:46:44] raises concerns about the sustainability and the environmental impact. So now moving on to the next one, we have limited problem-solving capabilities. While generative AI is great at tasks
[1:46:58] like writing essays or summarizing text, it's not well suited for the task requiring the deeper reasoning or the complex problem-solving. It can simulate
[1:47:10] problem-solving, but it doesn't truly understand the solution in the way humans do. Now moving on to the next one, we have risk of over-reliance. So with AI generating more and more content, there's a risk of becoming too
[1:47:27] dependent on the technology. So, people might rely on AI for the task that require human expertise or critical thinking, which could hinder the personal creativity or decision-making. So, now moving on we have data privacy
[1:47:43] issues. So, AI models can sometimes generate content based on the personal data, raising concerns about privacy. So, this is specially problematic in the So, this is specially problematic in the areas like healthcare, where sensitive
[1:47:57] data can be inadvertently shared or misused. So, now moving on to the next one, we have generalization challenges. So, generative models are great at specific tasks,
[1:48:10] but struggle with the broader generalization. For instance, a model trained to generate images of the cats may struggle to create the convincing images of a completely different animal, showing that AI is limited in its
[1:48:27] Now, moving on down the line, we'll understand the future of GenAI models. The first one is improved understanding and comprehension. So, the future of
[1:48:39] generative AI will likely see models that can understand the context more deeply, providing more relevant and meaningful outputs. So, we might see AI models that can truly comprehend the
[1:48:53] meaning behind the responses, rather than just spitting out the data. So, next we'll move on to the enhanced creativity. So, as AI models evolve, we
[1:49:05] can expect them to push boundaries and become more innovative. So, with advancements in the techniques like the reinforcement learning and neural architecture search, we could see AI producing truly original content,
[1:49:20] potentially revolutionizing fields like the art, music, and literature. So, now moving on, we have more ethical AI. So, in the future, we will likely
[1:49:32] AI. So, in the future, we will likely see more regulations around AI usage to prevent the harmful outcomes. So, AI companies are already working on improving the transparency, reducing bias, and making their models more
[1:49:47] ethical and responsible. So, now moving on, we have better So, now moving on, we have better personalization. So, the future holds AI that can offer more tailored personalized experience. So, imagine an
[1:50:00] AI that truly understands your needs and preferences. So, providing you with the content, products, or services that feel products, or services that feel personalized and relevant.
[1:50:13] Now, moving on to the next one, we have energy efficiency. So, as the demand for the generative AI grows, researchers are focusing on creating more energy-efficient models. We may see smaller, more powerful models that can
[1:50:28] run on less computational power, reducing the environmental impact. So, reducing the environmental impact. So, next we have human-AI collaboration. So, rather than replacing humans, the future of generative AI models is likely to be
[1:50:42] centered on collaboration. So, you will see AI helping people be more productive, augmenting creativity, and providing insights that humans can use to make better decisions. So, now moving on to the next, we have
[1:50:58] integration with the new technologies. So, generative AI will continue to integrate with the new technologies like augmented reality, that is the AR, then the virtual reality, that is the VR, the blockchain, and and opening up new
[1:51:13] possibilities for the immersive experiences and the decentralized applications. So, now moving on, we have solving complex real-world problems. So, in the near future, we could see AI models
[1:51:27] working alongside scientists to tackle the big challenges like climate change, pandemics, and global food security. So, AI could help analyze data, predict trends, and offer innovative solutions to these pressing problems.
[1:51:44] to these pressing problems. So, now we'll see the next one we have more accessible AI. So, as technology advances, AI will become more accessible. Tools will be easier to use, and people
[1:51:58] from all the walks of life, not just the technical experts, will be able to harness the power of AI to create and innovate. So, while generative AI models are still evolving, their potential to transform the industries, enhance
[1:52:13] creativity, and solve the complex problems is undeniable. But, it's important to address their limitations and work towards creating more ethical, efficient, and responsible AI systems. So, the future is exciting, and we are
[1:52:28] just getting started. So, guys, congratulations on completing the GenAI models for beginners course by Simplilearn. You have now built a strong foundation on the generative AI models by learning what generative AI was, how
[1:52:45] it impacted the real world, then creating chatbots, creating your first GenAI model, and then finally closing it with the future of GenAI models. So, as you move forward, continue experimenting with your own data sets and refining
[1:53:01] your own pipeline to unlock the full potential of GenAI in creating smarter, context-aware GenAI systems and models. >> Here's a jaw-dropper to start with. Job
[1:53:13] postings for generative AI skills have exploded from just 55 in January 2021 to exploded from just 55 in January 2021 to nearly 10,000 by May 2025. That's a growth of almost 200 times in just 4 years.
[1:53:29] Generative AI engineers have gone from niche specialists to some of the most sought-after professionals in tech. In the US, average base salaries now sit around $116,000, with top earners making as much as
[1:53:44] $179,000. And that's before bonuses and stock options. So, if you're looking to break into this field, you need more than hype. You need the right skills. In this video, I'll
[1:53:59] take you through the top 10 must-know skills to become a generative AI skills to become a generative AI engineer in 2025 and beyond. Each skill is backed with real-world relevance, surprising data, and examples
[1:54:12] you can relate to. Let's dive in. First up, at number 10, Python programming. Python is the universal language of AI. It's clean, versatile, and has libraries like NumPy, Pandas, PyTorch, and TensorFlow that
[1:54:29] Pandas, PyTorch, and TensorFlow that make building AI systems possible. Imagine trying to train a large language model without Python. It's nearly impossible. Even the world's biggest AI projects, from OpenAI's GPT
[1:54:45] to Meta's Llama, are built with Python at their core. Here's the kicker. Almost every AI-related job posting in the US lists Python as a required skill. Not optional, required. That makes it the
[1:55:01] single most indispensable tool in your toolkit. At number nine, we have machine learning fundamentals. Generative AI might sound advanced, but at its heart, it's still powered by the basics of machine learning.
[1:55:16] This includes supervised and unsupervised learning, decision trees, regression models, and understanding critical concepts like bias, variance, and overfitting. Why is this important? Because without
[1:55:31] strong ML fundamentals, you'll struggle to explain why your generative model is producing certain results, or worse, failing. And trust me, in an interview, that's where many candidates lose out. But here's a data twist. The US Bureau
[1:55:47] But here's a data twist. The US Bureau of Labor Statistics projects a 32% increase in data science jobs between now and 2033. That's nearly four times faster than the average growth rate across all
[1:56:00] occupations. So yes, ML fundamentals are your entry ticket to a booming career. Number eight is deep learning and neural Number eight is deep learning and neural networks. These are the power engines of
[1:56:14] generative AI. Without them, we wouldn't have tools like ChatGPT or Stable Diffusion. You need to know how CNNs work for vision tasks, how RNNs process
[1:56:26] sequences, and most importantly, how transformers have revolutionized the transformers have revolutionized the entire field. Here's a wild stat. Job entire field. Here's a wild stat. Job postings for AI engineers shot up 143%
[1:56:40] in a single year. Companies don't just want someone who knows the terms, they want someone who can explain backpropagation, activation functions, and attention mechanisms with confidence. If you can do that, you're
[1:56:54] already ahead of the curve. At number seven, natural language processing, or This is all about teaching machines to understand, process, and generate human language. You'll need to master
[1:57:09] language. You'll need to master tokenization, embeddings, and how models like BERT and GPT handle context. Remember, language is messy. Sarcasm, slang, typos, and even cultural context all matter in NLP.
[1:57:26] And here's the surprise. The NLP market is projected to reach $43 billion by 2025. Salaries follow the trend. Mid-level NLP Salaries follow the trend. Mid-level NLP engineers can make up to $170,000,
[1:57:41] engineers can make up to $170,000, while top experts earn $231,000 So, if you can build systems that actually understand language the way humans do, you won't just be in demand, you'll be in the top salary bracket.
[1:57:55] Number six is computer vision and generative models. This is where AI creates images, 3D models, or even video content out of thin air. We're talking about GANs, diffusion models, and autoencoders. The
[1:58:11] technologies behind tools like Midjourney and DALL-E. Midjourney and DALL-E. And the demand is massive. Meta pays up to $230,000 for skilled computer vision engineers,
[1:58:23] for skilled computer vision engineers, while Tesla averages around $161,000 Why? Because visual AI isn't just for art. It powers autonomous cars, medical imaging, and augmented reality.
[1:58:39] In short, if you can make machines see and create, you'll always be in high Halfway down the list, we land on number five. Transformers and large language models. Transformers changed the game. They're
[1:58:54] what make ChatGPT, Claude, and Google's Gemini possible. Knowing how LLMs are trained, fine-tuned, and deployed is no longer a nice-to-have. It's mandatory. Here's a stat that proves it. US AI-related job
[1:59:11] stat that proves it. US AI-related job listings grew 25% year-over-year in Q1 listings grew 25% year-over-year in Q1 2025. And the median salary for AI roles 2025. And the median salary for AI roles is now close to $157,000.
[1:59:25] investing heavily in LLMs and the people who can work with them. At number four, we have prompt engineering. Think of this as the art and science of talking to AI models. Small changes in wording can completely
[1:59:40] transform the output of an LLM. This makes prompt engineers incredibly valuable. And here's the jaw-dropper. Salaries for prompt engineers in 2025 now range from $95,000
[1:59:54] to $270,000. Just a few years ago, this job didn't even exist. Now it's one of the fastest-growing roles in tech. If you can design prompts that are efficient, reliable, and reusable, you'll stand out
[2:00:10] in interviews and at work. Number three is data engineering and ML Ops. is data engineering and ML Ops. Why does this matter? Because even the best AI models are useless without clean, structured data and smooth
[2:00:27] deployment pipelines. That's where ML Ops comes in. Handling training workflows, monitoring models, and automating retraining when things go wrong. Here's the surprise fact. AI-related
[2:00:41] jobs are twice as likely to offer parental leave and three times more likely to include remote options compared to traditional tech roles. That shows how companies are competing to attract and keep AI talent.
[2:00:56] At number two is responsible AI and ethics. This is about making sure your AI isn't biased, unsafe, or harmful. Think of cases where AI misclassifies patients in healthcare, or where models
[2:01:10] generate offensive content. These aren't just technical issues, they're legal and reputational risks. By 2025, nearly 47% of large
[2:01:22] organizations reported having experienced at least one AI-related issue tied to bias, fairness, or privacy. That's why companies now look for engineers who can ensure their models
[2:01:34] are trustworthy and compliant. Being skilled in responsible AI makes you more than just a developer, it makes you a leader. And finally, the number one skill for 2025 is agility. The ability to keep
[2:01:51] learning and adapting. Generative AI is evolving faster than any technology in history. What you learn today might be outdated in 6 months. The best engineers
[2:02:04] are not just coders, they're constant learners, experimenters, and adapters. Here's the kicker. Generative AI job postings in non-tech roles grew seven
[2:02:16] times from 2022 to 2024. While broader IT roles saw a 35 time growth. That's proof that agility is the most valuable skill of all.
[2:02:28] Because no matter how the field shifts, adaptable engineers will always thrive. To wrap this up, the median salary for AI engineers in the US is now around
[2:02:41] $145,000, with top talent making $200,000 or more. >> We will begin with the agenda for MCP tutorial for beginners. So, firstly, we will have a quick
[2:02:56] So, firstly, we will have a quick briefing of what exactly is an MCP, or as it is called, multi-agent collaboration protocol. Followed by that, we will understand some basic protocols and standards in
[2:03:08] communication. Next, we will try to relate MCP to the understanding of protocols and standards. Followed by that, we will get started with our demonstration on developing a web scraper agent using MCP.
[2:03:22] And for that, we will be needing Node.js, Cursor AI, and Psycho API. So, I will be walking you through those demonstration steps one after the other, and this will be a very beginner-friendly tutorial and practical
[2:03:36] demonstration on MCP. Now that I've made myself clear with the agenda, let's get started with MCP. So, what exactly is MCP? MCP? As MCP on their webpage says, MCP is a
[2:03:50] Model Context Protocol, or Multi-Agent Collaboration Protocol, is an open-source protocol that standardizes how applications supply context to large language models. Similar to how USB-C unifies hardware connections, MCP offers
[2:04:05] a consistent way to integrate AI models with various data sources and tools. Now, let's go to the official documentation of MCP to have a broader documentation of MCP to have a broader overview. So, as they say, MCP is one
[2:04:19] such tool. If you remember N8N and all the AI tools and data sources that you used. For example, if you wanted an AI travel agent, as we have an already existing tutorial on our channel, you can go through it for a brief overview.
[2:04:33] So, you might be needing an email writing agent, and you might be needing an airport code searching AI agent, hotel searching AI agent, flight booking AI agent, and a lot of agents connected to one another through a complex network
[2:04:47] using an agent platform, right? Now, in this particular MCP, that will be minimized. So, you will have a standardized approach where you just eliminate all those boilerplate coding, agents manual search for, databases
[2:05:03] manual search for agents which are specifically designed for the purpose. You just explain your initiative, attach You just explain your initiative, attach the API keys, and MCP takes over and
[2:05:15] gives you a fully furnished and finished product within no time. That's the main approach of MCP. So, here you can see an overview image where we have the parts MCP host, clients, servers, local data sources, remote services, and local data
[2:05:30] local data sources or local knowledge in case if you wanted some open source, then you can also go ahead with that. So, here we have all the quick starts and guides and example and tutorials to
[2:05:44] how to begin with that one, and all the explore pages that you want to go ahead into core architecture, the resources available, prompts, tools, transport samplings, and a lot more. And in case if you wanted to contribute your
[2:05:59] dedicated resources for the community, you can also go ahead with that. And you can also have some support and feedback for MCP. Now that we have a brief understanding of MCP and still clueless about what exactly is MCP, then let's go
[2:06:14] ahead with some standards and protocols so that you can get a template, and you can relate MCP to that template and understand what's the role or importance understand what's the role or importance of MCP in AI agent development. Now,
[2:06:27] understanding the standards and protocols. Now, let us imagine that you get a team of people to work on a project of people to work on a project and you are the manager that leads and
[2:06:40] directs this particular team for a purpose, right? Now, the complex purpose, right? Now, the complex problem. No one is friend of anyone. Everyone is new to everyone on the team and each and every person in this team
[2:06:55] doesn't have a common language in between. For example, person A speaks Mandarin, person B speaks Hindi, person C speaks French, so on. There is no common language between any one of those. Now,
[2:07:08] imagine each and every one of these team members as an AI agent or a tool working for the purpose. In this scenario, you might have to hard code the ports of communication, instructions on how to do. You need to go on to each and every
[2:07:24] single AI agent and explain the stuff in a very dedicated and a specific way. This could take a very long time. In the same way, if you relate this to a person, then you might have to reach to every single person and speak to them in
[2:07:37] their own tongue and explain them the procedures and the steps to be taken to finish the task. Now, how would you do that? It would be a lot complex. Mandarin, you might not even know French or a different language and in that
[2:07:51] particular scenario, you might be stuck. What if there was one single language that everyone knew? For example, English. Now, you can get all of them English. Now, you can get all of them together and explain the stuff in one
[2:08:03] single call and close it. And what might be so wonderful is that you'll not even waste a lot of time and there is no hard coding, there is no manual effort there, right? In the same way, if you relate this to a web server, then you are the
[2:08:17] client, you have the server and you just let's say you're working on Facebook or Instagram, you try to post something, you try to write a command, you try to activity. And these particular activities will go through four
[2:08:31] different types of requests. One is the get request, put request, post request, and last is the delete request. Right? Now, if you are connected to a single server, this might not be as tough as possible. But if you are getting
[2:08:45] connected to multiple servers, then you might be requiring different protocols every single time. Now, this can be combination of two parts, HTTP and the URL that you're trying to get connected to, right? Now,
[2:08:58] to eliminate the efforts between multiple servers and multiple requests at one go, you have one single tool called as API. And the most popular API is the REST API, which has a standard way of
[2:09:14] communicating. So, here the REST API works as the language English that we discussed before. In the same way, it has maintained a standard where each and every web client can have his communication to his dedicated server or
[2:09:30] multiple servers in one go using one single language or medium or standard single language or medium or standard that is REST API. Without REST API and the standards and protocols, it might be very confusing and you might end up in a
[2:09:43] complex situation which might not yield in the type of result you're expecting. Now, if you can relate the development of multiple AI agents for a single of multiple AI agents for a single purpose to the web communication and a
[2:09:57] meeting, you also expect if there were a template for development of AI agents which could make the process less complex, just like REST API did in web service, and just like English did in a meeting meeting, then MCP comes into
[2:10:13] picture. MCP is your template for it, right? meetings, you have MCP. Earlier, without MCP, we
[2:10:25] used to have one tool called NA10, where you had all the tools connected to it, and all the LLMs programmed to it, and you had to do this hard-coded coding for each and every tool that you have included in the network, and each and
[2:10:40] every agent that you have related to it, you had to specifically instruct, write down steps, develop a persona to your AI agent, right? There was a lot of stuff. But, if you eliminate this particular complexity with one tool, then that
[2:10:55] would be your MCP. You just had to include MCP and tell your purpose, and your job would be done. Now that you have a brief understanding or a general overview of what exactly is MCP, now let's go ahead and build one. Now, the
[2:11:11] the project that we are building, which is a web crawler or web scraper, we might need the help of Node.js. So, firstly, we will install and set the environment for Node.js. Followed by that, we will be needing an editor. So,
[2:11:27] for that, we will be choosing the Cursor AI editor. And next, we will use an API. So, for that, we will be using the Firecrawl API. And lastly, we can get started with building one once we have all the
[2:11:42] required resources. So, let's get started. So, as we discussed before, we'll get started by installing Node.js into our operating system. So, we're working on Windows operating system. So, we might have to go ahead with Windows.
[2:11:56] So, I'll choose Windows, and the architecture will be the one which I'm working with. So, that would be X64. But, in case if you are working with uh different operating systems, say Mac OS, then you might have to select ARM or any
[2:12:09] suitable architecture that you can be compatible with. For Linux, you'll have And so on. So, since we're working with Windows, so Windows it is and x64 architecture. And click on that Windows installer MSI file, so it will be
[2:12:24] getting started to install. And you'll have to click on that Windows downloaded. While it is getting downloaded, let's go to the next web page and download and install the Cursor AI that we will be needing for this
[2:12:37] in, create an account and log in. And automatically it'll give you the version we're in Windows. It gave me a Windows operating system. Just sign up or log account, just log in to your account and confirm it's that you and followed by
[2:12:54] that, you'll have installed your Cursor AI. Choose your theme. I'll go with the And here you make sure that you have Cursor tab, control key agent, and you can also choose the edition option. I'll go with the privacy mode. In case if you
[2:13:09] want the other one, go ahead with that. And uh here you have a few review settings and there you go. Now you're on the new window for Cursor AI. And meanwhile, let's check on the Node.js installation. I think uh the wizard has
[2:13:24] already started installing the Node.js setup. So, this is the window. Let's quickly uh go through each and every instance that we have gone through the beginning. So, it started off with some
[2:13:40] packages that we would be needing. So, uh Yeah, Chocolatey and uh KB2999. Yeah. Further, we have And the last one is Python 13. Right? Version 3.13.3
[2:13:57] is approved. So far, so good. I think we can now get started with the next step. So, here we are on the Cursor AI. So, we might have to get started going through the Cursor AI settings to create a new MCP. So, there are quite a few methods
[2:14:12] that you can try out to reach to settings. One is you can go to the file preferences and then you have the cursor settings. You just can use that vertical list or you can go ahead with the search options as well. So, you can use that
[2:14:27] where you can just go to the search window and search for the settings. Here you can see MCP on the left window. So, let me also walk you through a few other steps. So, go to the search window and type down Cursor AI or cursor
[2:14:42] I think there has been a discrepancy but not to worry, we'll go with the regular approach where we go to the file, preferences, and cursor settings. And there you have the options on the left-hand side. Go to MCP and so far you
[2:14:58] is the first time you logged into Cursor AI. So, you just have to add new global MCP server which is available on the blue color button. Just click on that. And now you'll have mcp.json file which is completely empty. You'll have to
[2:15:14] write down a few hardcoded codes here which are not too complex related to the earlier ones that you've worked on in A10. So, you might need a few more tools here. So, go to the Fire Crawl website.
[2:15:28] Before that, let's go through some awesome awesome MCP servers GitHub link.
[2:15:44] related to the MCP. So, let's open a new window and open all these links. What is MCP clients, tutorials, community, legend, server implementations, framework, tips and tricks. Right, for further information you can go through
[2:15:58] all these web pages individually. So, back to the home page. Here you have everything you needed. The basic information about exactly is MCP and further you have some server implementations and if you scroll down
[2:16:12] so you have awesome MCP service this this where the beginning is and next you have server implementations aggregators art and culture coding execution right customer data platforms so here you have aggregators art and culture dedicated
[2:16:27] web spaces so and you have also cloud platforms that you can work along with and moving further you have code executions coding agents command lines as well and communication related stuff so you
[2:16:42] can go along and perform your operations customer data or platforms data platforms and much more let's close all these windows and get started with the next stage where we need an API right so for that the
[2:16:57] project that we're working on which is web crawler we will be needing an API web crawler we will be needing an API from one of the providers which is Firecrawl API okay that's the one Firecrawl MCP
[2:17:11] particular link here to generate an API first you might have to sign up with Firecrawl so before we sign up let's go through beginning so that's the template code that you'll be working on and also here
[2:17:28] we have a wide variety of uh uh crawl options media parsing options smart weight actions right reliability first and all those things and also we have the pricing details for now we'll go ahead with the free plan and uh yeah
[2:17:42] let me add down my details my email address and you might also have to add a address and you might also have to add a pass so let me type down something that's a good thing to have a really
[2:17:55] strong website so that's a sign you need to have strong passwords and also make sure that you don't share your passwords and the API keys that you generate using 5 crawl. Now that my password is a little too longer, I think it should be
[2:18:09] Yeah. It's accessible and is a successful account generation. Now you need to check your email and confirm that the everything. So once you are signed up, let's go
[2:18:23] through another website for generating our API keys. our API keys. So 5 crawl API, scroll a little bit.
[2:18:37] So here you'll find some documentation related to 5 crawl and all the details that you need to go through for API key generation. So 5 crawl and CP server.
[2:18:49] looking for. 5 crawl MCP server, that's the one. And here you have overview, content, tools and other resources that you're looking for. Now well this is the template server configuration that you'll be needing. So
[2:19:03] make sure that you have an eye on it. And now you will also have the details that you might have to go through. Like basically it'll have the steps that you need to follow in the content page. Click on content.
[2:19:17] There you go. So it has all the features and details that you need to go ahead with. And tools you have the crawling, extraction, scraping tools that you can work along with 5 crawl. And few comments here.
[2:19:30] Now yeah, the email has not been confirmed yet. Let's try to resend and reconfirm the email so that we just sign up or login and create our API keys as
[2:19:42] soon as possible. So there you go. I have verified my email in my phone. So I'll just enter the credentials and sign in. Successfully you are signed in. On the left hand side you can see the overview, playground, extract, activity
[2:19:57] logs, usage, API keys, and all the settings. So, here you can see my API key has been by default hidden, so I'll not visualize it now. So, you have two options, either view or copy. So, we will be copying it for the first time to
[2:20:09] be on the safe side. So, since we are new to this particular website, so we don't have any activities registered on this particular webpage. So, now we have the API keys. Now, let's go to the content page to
[2:20:22] verify the steps that we need to follow. So, we need to install NPM, and we need So, we need to install NPM, and we need to go to the Cursor AI settings and add a new MCP server, which we already did. So, we just need to copy the template
[2:20:36] code, and we just need to place it there. And also, don't worry about where exactly to place your API keys in the template. We have that place. So, we are back on the Cursor AI. Now, we will replace this template code with the code
[2:20:51] that we just copied. And remember, if you're using Windows, try CMD/C, okay? If you're not on Windows, you can eliminate that CMD/C. But, if you are working in Windows, then you definitely need that CMD/C. Okay,
[2:21:06] let's erase this and replace that. So, I'll name my tool as Fire Crawl, and I'll go to the new line, where I'll write down my command. And the command will be the code that I just copied from the Fire Crawl window. So, let me type
[2:21:22] down the command in double quotes in new line, and I'll paste the command here, the Fire Crawl window. Control V. is in CMD/C,
[2:21:35] if you are working on Windows, that you definitely need it. If you're not omit it. Now, there might be some concerns with the codes. Don't worry. Now, to cross-verify, you can just go back to the Fire Crawl homepage and
[2:21:50] cross-verify it with the template for configuration that they have made us available. Let's go back to the fire wall window and at the same time you can see your API key here. So, that's where you need to place your API key.
[2:22:02] So, here we are on the fire wall window. We need to copy the API key and place it in the place where they have mentioned your API key. So, if you have cross verified with the configuration code, that's fine. And now
[2:22:16] check the double quotes. So, that's the one you need to focus on. Let me kind of if there is a way to zoom it in. Let me try to zoom in.
[2:22:29] Yeah, that's the one. CMD/C. If you're on Windows, you definitely need this. If you're not on Windows, you can just choose to omit it. Now, let me just do a one last final check. I think I missed a double quote somewhere.
[2:22:43] Yeah, I need to remove it from this place and add it to the end. Okay, remove the double quote in that place. Now, it should be after MCP.
[2:22:55] And once everything is done, you can just quickly choose to save and close distance between the API key and the double ampersand places. That's fine. And saving is really simple. Choose those three dots, close, save. And once
[2:23:09] to you know, choose to close this particular mcp.json and get back to your agent. have uh your new server that you've just created in the cursor settings. But in a
[2:23:23] few times you get no tool connected, no server connected, or there might be some errors. So, in that case you just need to cross verify the NPM installation, Node.js installation, and check all the files that are there or not. But even if
[2:23:37] everything is installed and you can see you know, the versions available and if you're still seeing this or watching that particular error, then don't worry. Just close the cursor AI and restart it, or you can also choose the refresh
[2:23:49] So, I've closed and restarted it and the same procedure go to file, preferences, cursor settings and there you'll see the MCP models and there you'll find it right there. Correct? Now, close it and go to the window
[2:24:04] where you'll find that the server. Yeah, that's the one and here you choose agent instead of anything else. Not money, not ask. I choose the agent
[2:24:16] and now I'll write down a simple prompt. Search for Simply Learn on Google. Now, it should give me results which it finds on Google. Maybe it's a homepage, maybe it's a social media handle like LinkedIn or YouTube or Instagram, anything that
[2:24:29] Only the term Simply Learn. So, here you can see it has given me three different results. First one is Simply Learn website, second one is Simply Learn YouTube channel and the third one is LinkedIn page. So, let's get started
[2:24:42] with the official web page. So, click on the Simply Learn.com. Now, yeah, it's asking me permission open. Now, we're on Simply Learn homepage. Similarly, let's go back and select for
[2:24:54] the YouTube page. So, that's the one. Open again, get permissions. So, you're on the Simply Learn YouTube page. Now, I think it might ask for a login for LinkedIn. But anyways, let's try to check if it can
[2:25:08] open without a login in to LinkedIn or not. So, yeah, you can see I've logged in to the YouTube. Yeah, it is asking me to login. So, let's quickly login. It might take a couple of seconds.
[2:25:21] So, there you go. I've logged in to LinkedIn. Now, open. So, I can see Simply Learn's homepage getting loaded in a second. Yeah, there you go. On LinkedIn, we have opened the Simply Learn's homepage successfully.
[2:25:33] >> Imagine you are managing a global supply chain company and where you have to handle orders, shipments and demand forecasting. But unexpected issues arises where sudden shortages like transport delays and the
[2:25:46] changes in demand. So, instead of relying on manual adjustments, what if automatically? This AI wouldn't just suggest actions. It would decide, execute, and continuously improve its strategies.
[2:26:01] that said, guys, I welcome you all on our today's tutorial on what is Agentic AI. Now, let us start with understanding first the first wave of artificial intelligence, which was predictive
[2:26:13] analytics or we could say data analytics and forecasting. What exactly happened like predictive AI focused more on analyzing the historical data, identifying the patterns, and making forecast about the future events. And
[2:26:26] these models do not generate any new content, but instead it was predicting outcomes based on the statistical models and machine learning. Now, technically how used to work? So, basically what we had like we used to
[2:26:38] take uh Suppose this is the ML model, okay? So, this is taking a structured data, which could be like suppose any past user activity or it could be a transaction record or any sensor reading. For example, you can consider
[2:26:51] reading. For example, you can consider say Netflix users watch history, okay? It could be any movie genre, watch time, and the user rating. So, now after this, what we were basically doing is we were doing the feature engineering or
[2:27:04] doing the feature engineering or preprocessing, okay? engineering process, we were extracting key features like user watch time trends, preferred genres, and watch frequency. And we could also apply
[2:27:20] scaling, normalization, and encoding techniques to basically make data more usable for the ML model. Then, we were using the ML models. Suppose it could be a time series forecasting models like ARIMA, LSTM, and all those given
[2:27:35] algorithms, which was basically predicting the future movie preferences based on the historical data. And in the output, guys, Netflix AI recommends new shows or movie based on the similar user patterns.
[2:27:50] So, this is how exactly the Netflix model was working incorporating the machine learning model. So, this was exactly the first wave of AI. Now, let exactly the first wave of AI. Now, let us discuss about the second wave of AI.
[2:28:08] which was basically content creation and use of conversational AI. So, you know, LM models like ChatGPT became very much popular during the second wave of artificial intelligence. So, what exactly was happening like generative AI
[2:28:21] was taking input data and it was producing new content such as text, images, videos, or even code. And these models learn from patterns in large data sets and it was generating human-like outputs.
[2:28:41] technology was working. So, basically, first there was a data input. Okay? So, basically, any prompt from the user. So, suppose in the GPT, okay? So, I'll just suppose in the GPT, okay? So, I'll just open GPT all over here and say we are
[2:28:55] Suppose we are giving any new prompt. Say such as Say such as write an article Okay? So, this was our given prompt. And after this, what exactly was happening
[2:29:11] was tokenization and preprocessing. So, the input text, suppose which I have written all over here, write a article on AI. So, this text was basically split into smaller parts. For example, like uh you could consider certain thing like
[2:29:26] this. So, here you have here you have write as one, uh you know, and as next. And similarly, you could carry on, you know, for the other words. Then, what
[2:29:40] exactly used to happen that these words were, you know, converted into word embeddings, means the numerical vectors representing words like in a higher dimensional space. And then we used to perform neural network
[2:29:53] processing. So, here the LLM processes input such as attention mechanisms, okay, or you know, using these models like GPT-4, BERT, and Llama. And with the help of self-attention layers, they were understanding the context and they
[2:30:08] were predicting the next word. Okay. Now, as a result, you are getting output certain thing like this. So, which was basically a generative AI phase. So, this was, guys, our second evolution of AI.
[2:30:26] it is basically agentic AI or autonomous AI. Now, what is this, guys? So, the agentic AI actually goes beyond text generation. So, it integrates decision-making, action execution, and autonomous
[2:30:40] learning. These AI systems don't just respond to prompts, but they also independently plan, execute, and optimize the processes. you could understand something like this.
[2:30:53] this. So, input or receiving any goal. So, user provides any high-level instruction. For
[2:31:05] example, it could be like, say, optimize warehouse shipments for maximum It could something be like that. And unlike generative AI, which would generate text, agentic AI executes real-world actions. After this, what
[2:31:20] Suppose the prompt that we have given, like optimize warehouse shipments for maximum efficiency, then the next step would have been querying the databases. The AI would pull the real-time data from multiple sources. So, it could be
[2:31:32] traditional database like SQL or no SQL, where we are fetching inventory levels or shipment history. Then, it could be a vector database from where it is receiving some unstructured data like past customer complaints and all those
[2:31:45] things. Then, with the help of external APIs, it is connecting to like APIs, it is connecting to like forecasting services or fuel price APIs or supplier ERP systems. And these things are like present with this
[2:31:57] respect. Then, the third step was the LLM decision making. Now, after acquiring the database, the AI agent processes data through the LLM based reasoning engine. Example like decision rules applied like
[2:32:11] it could automate supplier restocking orders. Like if shipment cost is increasing, then it is rerouting shipments through cheaper vendors. And suppose also if weather condition impacts the route, then it is adjusting
[2:32:25] the delivery schedules. Now, you could understand how agentic AI is behaving process. Now, next step would be action execution via APIs. So, AI is executing task without human intervention. It is
[2:32:40] triggering an API call to reorder stock from a supplier or update the warehouse robot workflows to prioritize fast-moving products or even send emails and notifications to logistic partners and about the changes what is going to
[2:32:54] be happening. And after this, finally, it is continuously learning, which is a data flywheel all over here. Okay? The AI is monitoring the effectiveness of its action like it was restocking efficient or did routing shipments, you
[2:33:09] know, reduce the cost and all. So, it is mon- monitoring the effectiveness of the action it has taken. And the data flywheel is continuously improving the future decisions. So, basically, it is using reinforcement learning and
[2:33:21] fine-tuning to optimize its logic. Okay. Now, let's have a just quick recap about the comparison of the all these three waves of AI. So, basically, predictive AI's main focus was on forecasting the trends, okay? While
[2:33:36] generative AI's was creating the content, and agentic AI, on the other hand, which is at the final step right now, is making decision and taking action. So, you could see how the evolution happened of AI in all these
[2:33:49] stages. And if you understand about the learning approach, then predictive AI was basically analyzing the historical data. While generative AI was learning from the patterns, like using text, image generation, okay? And but agentic
[2:34:02] AI is basically using the reinforcement its learning approach. Now, if you just look at the user involvement in predictive AI, so human is asking for the forecast and all. Here, human is
[2:34:16] giving the prompts. But, in the agentic AI, the prompts uh the intervention of human input has become very much minimal. If you could understand the technology, like basically, predictive AI was using
[2:34:28] machine learning, time series analytics. So, these kind of, you know, algorithms they were using. Generative AI was using transformers, like GPT, Llama, BERT, and all those things. Now, agentic AI is doing what, guys? It is
[2:34:41] using LLM plus APIs plus autonomous execution. So, we have discussed how this workflow is, you know, in a short way, how it is working. And moving ahead, we are also going to discuss through an example how exactly all these
[2:34:55] steps, like agentic AI, is working. So, based on the example, you could predictive AI, you know, Netflix recommendation model which they have on their system. And similarly, if we talk about
[2:35:09] about ChatGPT, you know, writing articles and all those things. And articles and all those things. And agentic AI, we could imagine like how AI if incorporated in supply chains, how you know things are working out. So
[2:35:22] guys, I hope so you would have got a brief idea regarding the three waves of AI. Now, let us move ahead and a bit understand about what is exact agentic AI. Now guys, let us understand the
[2:35:37] difference between generative AI and agentic AI. So, let us first, you know, deep dive into what exactly is a generative AI, okay? So, as you can see all over here that generative AI models generally are taking input query, okay?
[2:35:52] And they are processing it using LLM or large language model, and basically returning a static response without taking any further action. So, in this taking any further action. So, in this case, for example, a chatbot like
[2:36:04] uh ChatGPT, you know, it is taking the input from the user. So, as I've shown you earlier that uh say, suppose I've given an input like write a blog post on AI in healthcare. So, when I have written this given you know, user input
[2:36:18] or given the query, so when it goes to the large language model, this model is actually, you know, tokenizing all these input query, and it is retrieving the relevant knowledge from its training data. And it generate
[2:36:30] text based on the patterns. Now, we give the prompt, then LLM processes it, okay? And then we are getting the given output. So, now this is basically how,
[2:36:42] you know, generative AI is working. So, you could see all over here, we have GPT model, we have DALL-E, we have Codex. So, these are some of the, you know, amazing, you know, generative AI models, okay?
[2:36:55] Now, let us discuss a bit about DALL-E, which is actually a, you know, realistic image generation, you know, GenAI. So, like DALL-E is described as, you know, the realistic image generation model by the OpenAI. And this actually is a part
[2:37:09] of, you know, generative AI category alongside with GPT, which is basically for human-like language creation purposes. This model was created. And you could have also Codex for like it could be used for advanced code
[2:37:23] generation purposes. Now, let us discuss a bit about DALL-E. So, DALL-E is like a deep learning model basically, which is designed to generate realistic images from the text prompt. And it can create highly detailed and creative visuals
[2:37:36] users. So, some of the aspects of DALL-E like you could have all over here like text to image generation where users can input text prompts and DALL-E can generate unique images based on those
[2:37:51] description. The images generated by DALL-E are highly realistic and creative, okay? And it can generate photorealistic images, artistic illustration, and even surreal or imaginative visuals. They would also
[2:38:04] have customization and variability where it is allowing variation of an image edits based on text instruction and multiple style. So, this is also part of a generative AI model and it is This tool is actually
[2:38:17] playing a very amazing role. So, I will show you one example like how generative AI is actually working in the image generation purposes. So guys, as you can see all over here, I have opened this generative AI tool called DALL-E. Let us
[2:38:31] give a prompt to DALL-E and let us see how the image is generated. So, let's say we want a futuristic city
[2:38:44] filled with neon skyscraper and holographic billboards.
[2:39:08] robots. And we can have people wearing let's just say high-tech, you know, let's include some technology.
[2:39:22] uh DALL-E is trying to create a image. So, this is how actually generative AI is working. So, output comes up. Now, you could see all over here
[2:39:36] that uh this image which is generated, basically, this is generated by AI and you could see based on our prompt, it has given like the kind of, you know, based on this. Now, so this is one of the amazing
[2:39:53] this, guys. Okay? Now, guys, let us discuss about agentic AI or autonomous decision-making and action execution. So, you could see this diagram all over here. So, agentic AI like unlike the
[2:40:07] generative AI, it is not generating responses, but it is also executing a task autonomously based on the given query. For example, like if you take AI in managing a warehouse inventory, okay? Suppose we want to optimize the
[2:40:21] warehouse shipment for the next quarter. So, here what is going to happen? So, first the agent is going to receive its goal all over here, okay? And this AI agent, you know, is going to query the external
[2:40:34] data sources. So, it could you know, for example, it could be your you know, inventory databases or logistics API. And then it retrieves real-time inventory levels and it demands the given forecast. Okay. Now,
[2:40:49] at here it is going to make the autonomous discussions and the kind of output we are going to get will be kept in observation by this agent, okay? So, basically, it is going to analyze the current warehouse stock,
[2:41:02] predict demand for the next quarter, check the suppliers availability, and automate the restocking if inventory is below the given threshold. So, uh for example, you could imagine uh you know, suppose based on the you know,
[2:41:17] output what we are going to get all over here. So, based on this output, we could here. So, based on this output, we could get certain thing like this. Like say, current inventory level
[2:41:32] inventory level like say, 75% percent capacity. it could have also other thing like say,
[2:41:45] say, demand forecast say, 30%
[2:41:57] expected in uh quarter two. And also, it is going to go say, like And also, it is going to go say, like say, reordering
[2:42:10] initiated. So, this is the output what we are going to get based on the supply chain management in example what we are trying to get. So, as we have seen in generative AI, user is giving the input,
[2:42:23] okay, prompt, then it is using LLM model to generate the given output. But, agentic AI is doing what, guys? It is going It is going to take an action, you know, beyond just generating a text. So, in this scenario, it is querying the
[2:42:36] inventory databases, it is automating the purchase order, it is going to select the optimal shipping providers which could be, you know, suitable for the given company. It is going to continuously refine the strategies based
[2:42:48] on the real-time feedback. So, guys, let's recap once more. So, if you talk about the function base, then GNA is more concerned with producing a written content or a visual content, okay? And even it can code from the pre-existing
[2:43:02] guys, decision-making, taking actions towards a specific goal, and it is focused on achieving the objectives by interacting with the environment and making the
[2:43:16] autonomous decision. GenAI is exactly relying on the existing data to predict and generate content based on say patterns it has learned during its training phase. But it does not adapt or evolve from its experiences. Whereas if
[2:43:29] I talk about agentic AI, it is adaptive. So it is learning from its actions and experiences. It is improving over time by analyzing the feedback, adjusting its behavior to meet objectives more effectively. With the help of GenAI,
[2:43:42] human input is essential to the prompt so that, you know, basically with the help of that it could go into the LLM model and it could generate the given, you know, output based on your prompt. Once
[2:43:56] you set up the agentic AI, it requires like minimum human involvement. It operates autonomously, making decisions and adapting to changes like without continuous human guidance. And it can even learn in real time. So that's with
[2:44:10] the beauty of agentic AIs. So we have given one example of GenAI like basically giving prompt to the ChatGPT or DALL-E, okay? And agentic AI one example could be your supply chain management system. Now, let us bit deep
[2:44:26] dive into understanding the technical aspects of how agentic AI is exactly working. Now, guys, let us try to understand how agentic AI is exactly working. So there is actually a four-step process of, you know, how
[2:44:42] four-step process of, you know, how agentic AI exactly works. So the first step is, you know, perceiving. Where basically what we are doing is we are gathering and processing information from databases, sensors, and digital
[2:44:55] environments. And also, the next step is reasoning. So, with the help of large language model as a decision-making engine, it is generating the solutions. If we talk about the third step, which is acting, so it is integrating with
[2:45:08] external tools and softwares to autonomously execute the given task. And finally, it is learning continuously to improve through the feedback loop, which is also known as the data flywheel. Okay. Now, let us explore each of the
[2:45:22] step one by one and let us try to understand. So, if you talk about perceiving, okay? So, this is actually the first step where agentic AI is actually stepping up. So, it is doing the perception, where what exactly is
[2:45:35] happening, guys? That AI is collecting data from multiple sources. So, this data could be from database, okay? Like your traditional and vector databases, okay? So, it could be graph QL like vector database means the same. And if
[2:45:51] we talk about other from data, it could be from APIs like it is fetching real-time information from external systems. It is basically taking data from the IoT sensors like for real-world applications
[2:46:03] like robotics and logistics. And also, it could take, you know, data from the user inputs also. Like it could be text command, voice commands, or a chatbot interaction. Now, how it is exactly working, guys? So, basically, let us
[2:46:18] recollect everything technically and let us see how this is happening. So, the first step which is going in perceiving is the data extraction, where exactly the AI agent queries the structured databases like SQL or NoSQL
[2:46:32] for relevant records. It is also using vector databases to retrieve any semantic data for context-aware responses like it could anything, you know, it is trying to find out.
[2:46:45] Okay. So, next, after it has got the data extraction, it goes for feature extraction and pre-processing where AI is filtering the relevant features from the raw data. For example, like a fraud
[2:46:57] detection AI is scanning the transaction log for anomalies. The third thing it is entity recognition and object detection. So, AI uses basically computer vision to detect objects and images and then it is applying the named entity
[2:47:13] recognition. This is a technique, okay? Uh to extract the critical terms from the given text also. So, we have three uh step-by-step process which is happening in uh perceiving. The first one is data extraction, second one is a
[2:47:27] feature extraction and pre-processing, the third one is like entity recognition and object detection. So, uh let us take a very simple example like uh AI-based customer support system. So, if we consider an agentic AI assistance like
[2:47:42] for a customer service. So, say a customer is asking, "Where is my order?" So, the AI queries multiple databases all over here. Suppose, it is going to query the e-commerce order database to retrieve the order status or it could go
[2:47:55] to the logistics API to track the real-time shipment location. Also, it history to provide personalized response. The result what we get all over here is that the AI is fetching the tracking details, identifying any delays
[2:48:09] if it is happening, and suggesting the best course of action. Now, uh the next best course of action. Now, uh the next step is reasoning, okay? Now, AI's understanding and decision-making and problem-solving is making agentic AI
[2:48:22] greater. So, here what is exactly happening like once the AI has perceived the data, now it should start reasoning it, okay? So, the LM model acts as a orchestrating AI processes and integrating with a specialized models
[2:48:39] for various function. So, if you talk about the key components like here used in the reasoning, it could be LLM based decision-making. So, AI agents could use models like LLMs, like GPT-4, Cloud, Llama to interpret a user intent and
[2:48:55] generate a response. It is basically coordinating with smaller AI models for domain-specific tasks. Like it could be like financial prediction or medical diagnostics. So, these could be you know given example. Then it is using
[2:49:09] a retrieval augmented generation or RAG model, okay? To with the help of which AI is enhancing the accuracy, you know, by retrieving any proprietary data from the company's databases. For example, like instead of relying on GPT-4's
[2:49:23] knowledge, the AI can fetch company-specific policies to generate the accurate answers. So, this could be the one. And in in the reasoning, the final step is AI workflow and planning. So, this is a multi-step reasoning where
[2:49:37] AI is breaking down complex task into logical step. For example, like if asked to automate a financial report, AI is retrieving the transaction data, analyzing the trend, and it is formatting the result. Also, for
[2:49:49] example, you could use this in supply chain management. So, suppose consider there is a logistics company which is using the agentic AI to optimize what could be the you know shipping routes, you know. So, a supply
[2:50:02] chain manager requesting the AI agent to find the best shipping route to reduce the delivery cost. So, the AI processes real-time fuel prices, traffic conditions, and weather report. So, using LLM plus data retrieval, it finds
[2:50:16] out the optimized routes and selects the cheapest carrier. Result you get is that AI chooses the best delivery option. So, here the cost is reduced and improving efficiency. So, this is one of the use cases, guys.
[2:50:30] So, after perceiving you get is reasoning. Okay. Now, let us move ahead and discuss about the third step, which is act. So, in this step, basically, what is happening like AI is taking autonomous actions.
[2:50:43] So, unlike generative AI, which stops at generating content, so agentic AI takes the real-world action, okay? How AI is executing tasks autonomously, guys? So, basically, first step is like here the integration with APIs and software could
[2:50:57] be happen, where AI can send automated API calls to the business systems. For example, like reordering the stock from the supplier's API. So, suppose any inventory level is going down, so it could, you know, reorder that particular
[2:51:12] stock from the supplier's API. So, it is interacting with the given API. Now, it could also automate the workflows like AI executes multi-step workflows without human supervision. So, here like AI can handle like insurance claims by
[2:51:27] verifying the documents, checking policies, and approving the payouts. And finally, AI could operate within predefined business rules, okay? To prevent any unauthorized actions, also. So, ethical AI is a basically being
[2:51:41] worked in this direction. For example, like AI can automatically process claims 10,000 US dollars, you know, but it is requiring the human approval for the higher amounts. So, based on, you know,
[2:51:55] insurance and policy-making stuff, so agentic AI could be, you know, really helpful in this scenario. One example, like let us consider. So, let's say we have this agentic AI managing an IT support system. So,
[2:52:09] suppose a user says, "My email server is down." So, the AI can diagnose the issue, restart the server, and confirms the given resolution. Now, if it is unresolved, then AI escalates to a human technician. Then, it results into, you
[2:52:25] know, AI is fixing the issues autonomously, reducing the downtime. Okay. So, this is where your action or act is coming up into the picture. Now, if you go on to the next and final step, which is learning. So, learning
[2:52:39] basically, with the help of data flywheel, it is continuously learning. Okay? So, this is the feedback loop all over here, which is the data flywheel. So, how AI learns over the time? If we ask this question, so what is exactly
[2:52:53] happening that it is interacting with the data collection. Suppose, AI logs successful and failed actions. For example, like if users correct AI-generated responses, the AI is learning from those corrections. Second
[2:53:07] thing what you could do is you could model you could fine-tune the model and do reinforcement learning. So, AI adjusts its decision-making models, you know, basically to improve future accuracy. It uses reinforcement learning
[2:53:20] basically to optimize workflows based on past performance. Okay? Now, third step could be automated data labeling and self-correction. So, here what is happening that AI is labeling and categorizing past interactions to refine
[2:53:34] its knowledge base. Example, like AI autonomously is updating frequently asked answers based on the recurring user queries. So, in this way AI is learning over the time. Example one you could consider so say we
[2:53:48] have this AI is optimizing any financial fraud detection. So, say this is consider that this is a bank which is AI-powered which has this AI-powered fraud detection system. So, AI is
[2:54:01] analyzing these financial transaction and it is detecting any suspicious activity. And if flagged, the transactions are false AI is learning to reduce these false alerts. So, over the time AI is
[2:54:13] improving the fraud detection accuracy like minimizing disruptions for the customer. So, in this way AI is getting smarter over the time like reducing the false alerts and also the financial fraud. So, let's have a just quick recap
[2:54:26] of what we studied right now. So, agentic AI works in four steps. The first step is perceiving where AI is gathering data from databases, sensors, and APIs. The next step is reasoning.
[2:54:39] So, it is using LLM to interpret task, applies logic, and generating the solution. The third step is acting. So, here AI is integrating with external systems and automating the task. And finally, it is learning. So, AI is
[2:54:52] improving over the time, you know, via feedback loop or which is basically called as data flywheel. So guys, now let us see this diagram and try to understand what this diagram is trying to say. So, the first thing you could
[2:55:06] see an AI agent all over here. So, this is an AI agent which is basically an autonomous system. So, which has the capability of perceiving its environment, making decision, and executing actions without any human
[2:55:18] intervention. Now, AI agent is acting as the central intelligence, okay, in this given diagram. And it interacts with the user, okay, and various other data sources. It processes input, queries databases,
[2:55:32] makes decision using a large language model. And it is executing action, and it is learning from the given feedback. Now, the next step you could see the LLM model. So, if we talk about LLMs, these are the large language model which is
[2:55:46] kind of an advanced AI model trained on massive amount of text data to understand, generate, and reason over natural language. Now, if I talk about this LLM, so this is actually acting as the reasoning engine all over here. And
[2:56:01] it is interpreting the user inputs and making informed decision. It is also retrieving relevant data from the databases, generating responses. It can also coordinate with multiple AI models for different task. Like it could be
[2:56:14] content generation, okay, predictions, or decision-making. Now, when the user is asking a chatbot, like for example, let's say, what is my account balance? So the LLM processes the query, retrieves the relevant data, and
[2:56:28] responds the given bank balance accordingly. But now if you look at the kind of database the LLM is interacting, so we have the traditional database and the vector database. So here, if I say
[2:56:41] the database like AI agent basically is querying the structured database. So suppose structured database like it could be a customer records or inventory data or it could be any transactional log also. So traditional databases
[2:56:54] basically store well-defined in a structured information. Okay. So for example, like when a bank AI assistant is processing a query like show my last five transaction, so it is basically fetching the information from a
[2:57:06] traditional SQL-based database. Next, we have this vector database also, guys. So vector database is a specialized kind of a database for storing unstructured embeddings, images, or audio representations. So guys, like unlike
[2:57:22] traditional databases that store exact values, vector databases store in a high-dimensional mathematical space. It allows AI models to search semantically similar data instead of like exact matches. Now AI is retrieving the
[2:57:36] contextual information from the vector databases, which is actually enhancing the decision making. It is improving the AI memory by allowing the system also to search for, you know, conceptually similar past
[2:57:48] interaction. Let us take an example to understand this. For example, we have discussed about a customer support chatbot. So suppose if it queries a vector database to find out similar past tickets like when responding to a
[2:58:01] engine could use a vector database to find out similar products on a user's past preferences. So this could be done in that scenario.
[2:58:13] Also, some of the like popular vector databases could be like Facebook's AI databases could be like Facebook's AI similarity search, Pinecone or Weaviate. These are the certain uh amazing uh vector databases. Then you could see the
[2:58:25] next step is, you know, after it has worked on these given data, it is performing the action. So, the action component is referring where AI's agent has this ability to now execute task autonomously after the reasoning is
[2:58:40] done. So, AI is integrating with external tools, APIs, or automation software to complete the given task. It does not provide only information, but it is actually uh say, you know, performing the given action. So, for
[2:58:53] example, like in a customer support, the AI can automatically reset a user's password after verifying the identity. If you talk about in finance, then AI can approve a loan also, like based on the predefined eligibility criteria.
[2:59:07] Now, finally, we have the data flywheel. So, data flywheel is a continuous feedback loop where AI is learning from the past interactions, refining its models, and it is always improving over the time. Now, every time like the AI is
[2:59:21] interacting with the data or taking an action or receiving a feedback, that information is fed into this model. So, this is creating a self-improving AI system that is becoming smarter over the time. So, the data flywheel is
[2:59:35] allowing AI to learn from every interaction, and AI is becoming more efficient by continuously optimizing responses and refining strategies. Best thing in could be used in a fraud detection. So, in this, the AI is going
[2:59:50] to learn from the past fraud cases, and it is going to detect new fraudulent patterns and more effectively. Chatbots also can learn from user feedback and improve the responses. And finally, you have the model customization, which is
[3:00:04] basically you are trying to fine-tune the AI models on specific business need or any industry requirement. So, AI models are not static, like they can be adapted and optimized for a specific task. So, custom fine-tuning is
[3:00:17] actually improving the accuracy and domain-specific application, like it cybersecurity. So, a financial institution, uh say, fine-tuning an LLM to generate an investment advice, okay,
[3:00:30] on a historical market trends. That could be one use case. Or in healthcare, if we discuss, like uh the healthcare provider is fine-tuning then AI model to interpret the medical reports and recommend the treatments. So, guys,
[3:00:43] based on the given diagram, you would have got a brief idea like how, uh you have got a brief idea like how, uh you know, agentic AI is working. Now, if we discuss about the future of agentic AI, then, guys, I would say it looks very
[3:00:56] much promising because it is keep improving itself and it is finding new ways to be useful. Like with better machine learning algorithms and smarter decision-making, these AI system will be more uh independent handling complex
[3:01:10] tasks on their own. And believe me, in industries like healthcare, finance, customer service, they have already started to see how AI agents can make more impact and it could be more efficient from personalization
[3:01:23] perspective, you know, managing resources, and many more other things. So, as the system continue to learn and adapt, I think so they will be opening up even more possibilities, helping businesses grow, improving how we live
[3:01:36] businesses grow, improving how we live and work. Now, I would say that uh in conclusion that agentic AI is actually paving the way for new opportunities, which was assisting with generating content or predicting the data, you
[3:01:50] know, uh responding to any queries, but agentic AI can perform techniques independently with minimal human effort. And agentic AI has become self-reliant in decision-making way, and it is making a very big differences in industry like
[3:02:05] healthcare, logistics, customer services, which is enabling companies to be more efficient. As a result, it is providing better services to their providing better services to their clients.
[3:03:18] machine learning, so let's see if we can give a little bit more of a concrete definition. What is machine learning? Machine learning is a science of making computers learn and act like humans by feeding data and information without
[3:03:31] being explicitly programmed. We see here we have a nice little diagram where we have our ordinary system, and your computer nowadays you can even run a lot phones have advanced so much. And then
[3:03:44] with artificial intelligence and machine learning, it now takes the data and it learns from what happened before, and then it predicts what's going to come next, and then really the biggest part right now in machine learning that's
[3:03:56] going on is it improves on that. How do we find a new solution? So, we go from stuff and understanding how it fits together to predicting what it's going to do to post scripting, coming up with a new solution. And when we're working
[3:04:11] on machine learning, there's a number of different diagrams that people have posted for what steps to go through. A lot of it might be very domain specific. So, if you're working on photo identification versus language versus
[3:04:26] medical or physics, some of these are switched around a little bit or new to the domain. This is kind of a very general diagram. First, you want to know what it is you're wanting to predict. Then you're going to be
[3:04:40] defined an objective, you need to collect the data that matches. You spend a lot of time in data science collecting data. And the next step, preparing the data is clean going in. There's the old saying, bad data in, bad answer out or
[3:04:57] bad data out. And then once you've gone through and we've cleaned all this stuff coming in, then you're going to select the algorithm. Which algorithm are you algorithm. In this case, I think we're going to be working with SVM, the
[3:05:11] support vector machine. Then you have to test the model. Does this model work? Is this a valid model for what we're doing? And then once you've tested it, then you want to run your prediction. You want to run your prediction or your choice or
[3:05:23] whatever output is going to come up with. And then once everything is set you want to go ahead and deploy the model. And remember I said domain specific, this is very general as far as the scope of doing something. A lot of
[3:05:36] models you get halfway through and you realize that your data is missing new data because you've run a test in here someplace along the line. You're answers I need." So, there's a lot of things that are domain specific that
[3:05:49] become part of this model. This is a very general model, but it's a very good model to start with. And we do have some basic divisions of what machine learning instance, do you want to predict a category? Well, if you're categorizing
[3:06:03] thing, that's classification. For instance, whether the stock prices will increase or decrease. So, in other words, I'm looking for a yes-no answer. Is it going up or is it going down? And in that case, we'd actually say, is it
[3:06:15] going up, true? If it's not going up, it's false, meaning it's going down. This way, it's a yes-no, 0-1. Do you want to predict a quantity? That's regression. So, remember we just did classification, now we're looking at
[3:06:27] regression. These are the two major divisions in what data is doing. For instance, predicting the age of a person based on the height, weight, health, and different factors, you might guess how old a person is. And then there are a
[3:06:40] lot of domain-specific things. Like, do you want to detect an anomaly? That's anomaly detection. This is actually very popular right now. For withdrawal anomalies. You want to know when someone's making a withdrawal that
[3:06:53] might not be their own account. We've actually brought this up because this is predicting the stock, whether to buy stock or not, you want to be able to market is an anomaly, use a different
[3:07:06] is going on, you got to pull out new information in there, or is this just the norm? I'm going to get my normal return on my money invested. So, being able to detect anomalies is very big in data science these days. Another
[3:07:19] question that comes up, which is on what we call untrained data, is do you want to discover structure in unexplored data? And that's called clustering. For instance, finding groups of customers with similar behavior given a large
[3:07:34] database of customer data containing their demographics and past buying records. And in this case, we might notice that anybody who's wearing certain stores. Or whatever it is, they're going to make certain purchases.
[3:07:48] By having that information, it helps us to market or group people together so that we can now explore that group and find out what it is we want to market to And that might also work in just about any arena. You might want to group
[3:08:01] people together whether they're based on their different areas and investments and financial background whether you're going to give them a loan or not before you even start looking at whether they're valid customer for the bank, you
[3:08:15] different areas and group them together based on unknown data. So you know you tell you but you want to cluster people together that come together. Let's take a quick detour for quiz time. Oh, my favorite. So we're going to have
[3:08:29] a couple questions here under our quiz time and we'll be posting the answers in these part two of this tutorial. So let's go ahead and take a look at these you'll get them all right. It'll get you thinking about how to process data and
[3:08:44] what's going on. Can you tell what's happening in the following cases? Of cup of coffee and you have your check box and your pen trying to figure out what's your next step in your data science analysis. So the first one is
[3:08:57] grouping documents into different categories based on the topic and categories based on the topic and content of each document. Very big these days. You know you have legal documents, you have maybe it's a sports group
[3:09:10] documents, maybe we're analyzing newspaper postings. But certainly having that automated is a huge thing in today's world. B, identifying handwritten digits in images correctly. So we want to know whether
[3:09:24] they're writing an A or capital A, B, C. What are they writing out in their hand digit of their handwriting. C, behavior of a website indicating that the site is not working as designed. D, predicting salary of an individual based on his or
[3:09:40] her years of experience. You know HR hiring set up there. So stay tuned for these questions when we get to the part two of this tutorial, or you can just simply write at the bottom and send a note to Simply Learn and they'll follow
[3:09:55] up with you on it. Back to our regular content. Now, these last few bring us into the next topic, which is another way of dividing our types of machine learning, and that is with supervised, unsupervised,
[3:10:08] and reinforcement learning. Supervised learning is a method used to enable machines to classify and predict objects, problems, or situations based on label data fed to the machine. And in here, you see we have a jumble of data
[3:10:22] with circles, triangles, and squares, and we label them. We have what's a circle, what's a triangle, what's a square, and we have our model training, answer. Very important when you're doing supervised learning, you already know
[3:10:35] the answer to a lot of your information coming in. So, you have a huge group of data coming in, and then you have new data coming in. So, we've trained our model, the model now knows the difference between a circle, a square, a
[3:10:47] triangle. And now that we've trained it, we can send in, in this case, a square that the top one's a square, and the next one's a circle. And you could see that this is uh being able to predict whether someone's going to default on a
[3:11:00] earlier. Supervised learning on stock market, whether you're going to make money or not. That's always important. And uh if you are looking to make a fortune in the stock market, keep in mind, it is very difficult to get all
[3:11:13] the data correct on the stock market. It is very uh it fluctuates in ways you really hard to predict. So, it's quite a roller coaster ride if you're running you start realizing you really have to dig for new data. So, we have supervised
[3:11:27] learning, and if we have supervised, we should need unsupervised learning. In unsupervised learning, machine learning model finds the hidden pattern in an unlabeled data. So, in this case, instead of telling it what the circle
[3:11:40] is, and what a triangle is, and what a square is. It goes in there, looks at groups them together. Maybe it'll group it by the number of corners. And it notices that a number of them all have three corners, a number of them all have
[3:11:53] four corners, and a number of them all have no corners. And it's able to filter We talked about that earlier with looking at a group of people who are out shopping. We want to group them together to find out what they have in common.
[3:12:06] And of course, once you understand what people have in common, maybe you have store, or you have five of them are customer at your store, and they have a lot in common with five others who are not customers at your store. How do you
[3:12:18] customers at your store yet? They fit the demographics of who's going to shop your store, not the one next door. Of course, this is a simplified version. between a triangle and a circle, which
[3:12:31] is might not be so easy in marketing. Reinforcement learning. Reinforcement learning where an agent learns how to behave in an environment by performing actions and seeing the result. We have here where the in this case, a baby.
[3:12:45] infant for this slide, cuz the reinforcement learning is very much in its infant stages. But it's also probably the biggest machine learning future. It's going to be coming up over the next few years is reinforcement
[3:13:00] learning and how to make that work for us. And you can see here where we have our action. And the action in this one, it goes into the fire. Hopefully the baby didn't It was just a candle, not a giant fire pit like it looks like here.
[3:13:12] And the baby comes out, and the new state is the baby is sad and crying, cuz they got burned on the fire. And then maybe they take another action. The one taking the actions. And in this case, they didn't go into the fire. They
[3:13:25] baby's happy and laughing and playing. Reinforcement learning is very easy to understand, cuz that's how as humans, that's one of the ways we learn. We learn whether it is, you know, you burn yourself on the stove, don't do that
[3:13:38] anymore. Don't touch the stove. In the big picture, being able to have machine learning program on or AI be able to do this is huge cuz now we're starting to learn how to learn. That's a big jump in the world of computer and machine
[3:13:52] learning. And we're going to go back and just kind of go back over supervised versus unsupervised learning. Understanding this is huge because this is going to come up in any project you're working on. We have in supervised
[3:14:04] learning we have labeled data, we have direct feedback. So someone's already gone in there and said, yes, that's a triangle. No, that's not a triangle. And then you predict an outcome. So you have a nice prediction. This is this this new
[3:14:16] what it's going to be. And then with unsupervised training, it's not labeled. So we really don't know what it is. There's no feedback. So we're not we're not telling it whether it's a triangle or a square, we're not telling
[3:14:31] it to go left or right. All we do is we're finding hidden structure in the data. Grouping the data together to find out what connects to each other. And then you can use these together. So imagine you have an image and you're not
[3:14:45] sure what you're looking for, so you go in and you have the unstructured data, find all these things that are connected together. And then somebody looks at those and labels them. Now you can take that labeled data and program something
[3:14:57] to predict what's in the picture. So you can see how they go back and forth and different tools together to make a bigger picture. There are many interesting machine learning algorithms. Let's have a look at a few of them.
[3:15:10] of what's out there. And these are some of the most important ones that are currently being used. We'll take a look at linear regression, decision tree, and the support vector machine. Let's start with a closer look at linear regression.
[3:15:24] Linear regression is perhaps one of the most well-known and well-understood algorithms in statistics and machine learning. Linear regression is a linear model. For example, a model that assumes a linear relationship between the input
[3:15:37] variables X and the single output variable Y. And you'll see this if you remember from your algebra classes, Y = MX + C. Imagine we are predicting distance traveled Y from speed X. Our linear regression model representation
[3:15:52] linear regression model representation for this problem would be Y = M * X + C for this problem would be Y = M * X + C or distance = M * speed + C. Where M is the coefficient and C is the Y intercept. And we're going to look at
[3:16:04] two different variations of this. First, we're going to start with time is constant. And you can see we have a bicyclist. He's got a safety gear on, thank goodness. Speed equals 10 m/s. And so over a certain amount of time, his
[3:16:17] distance equals 36 km. We have a second bicyclist who's going twice the speed or bicyclist who's going twice the speed or 20 m/s. And you can guess if he's going then he's going to go twice the distance. And that's easy to compute. 36
[3:16:32] distance. And that's easy to compute. 36 * 2, you get 72 km. And so if you had the question of how fast would somebody who's going three times that speed or 30 m/s is, you can easily compute the distance. In our head, we can do that
[3:16:45] without needing a computer. But we want to do this for more complicated data. So But let's just take a look at that and what that looks like in a graph. So in a linear regression model, we have our distance to the speed and we have our M
[3:16:58] equals the V E slope of the line. And we'll notice that the line has a plus slope. And as speed increases, distance also increases. Hence, the variables have a positive relationship. And so
[3:17:10] your speed of the person which equals Y = MX + C, distance traveled in a fixed interval of time. And we could very easily compute either following the line or just knowing it's three times 10 m/s that this is roughly 102 km distance
[3:17:24] that this third bicyclist has traveled. One of the key definitions on here is positive relationship. So, the slope of the line is positive, as distance increase, so does speed increase. Let's take a look at our second example where
[3:17:38] we put distance as a constant. So, we have speed equals 10 m per second. They have a certain distance to go and it takes him 100 seconds to travel that bicyclist who's still doing 20 m per second. Since he's going twice the
[3:17:52] speed, we can guess he'll cover the distance half the time, 50 seconds. And of course, you could probably guess on the third one, 100 divided by 30, since he's going three times the speed, you can easily guess that this is 33.333
[3:18:06] seconds time. We put that into a linear regression model or a graph. If the let's see the relationship between speed and time. And as time goes up, the amount of speed to go that same distance goes down. So, now your m equals a minus
[3:18:21] v e slope of the line. As the speed increases, time decreases, hence the variable has a negative relationship. Again, there's our definition, positive relationship and negative relationship, dependent on the slope of the line. And
[3:18:35] with a simple formula like this, um and even a significant amount of data, let's implementation of linear regression. And have this data set where we have x y, x
[3:18:48] equals 1 2 3 4 5, standard series, and the y value is 3 2 2 4 3. When we take points on a graph, you can see there's
[3:19:00] probably eyeball a line through the middle of it. But we're going to calculate that exact line for linear regression. And the first thing we do is we come up here, we have the mean of x i. And remember, mean is basically the
[3:19:13] i. And remember, mean is basically the average. So, we added 5 + 4 + 3 + 2 + 1 and divide by 5. And that simply comes out as 3. And then we'll do the same for Y. We'll go ahead and add up all those numbers and divide by five and we end up
[3:19:27] with a mean value of Y of I equals 2.8. Where the XI references is an average or means value and the YI also equals a means value of Y. And when we plot that, you'll see that we can put in the Y equals 2.8 and the X equals 3 in there
[3:19:43] little different color so you can sort it out with the dash lines on it. And the linear regression, the linear regression model should go through that dot. Now, let's find our regression equation to find the best fit line. I
[3:19:58] equals MX plus C. So, we're looking for M and C. So, to find this equation for our data, we need to find our slope of M and our coefficient of C. And we have Y
[3:20:10] and our coefficient of C. And we have Y equals MX plus C, where M equals the sum of X minus X average times Y minus Y average or Y means and X means over the sum of X minus X means squared. That's how we get the slope of the value of the
[3:20:25] line. And we can easily do that by creating some columns here where we have XY. Computers are really good about iterating through data and so we can easily compute this and fill in a graph of data. And in our graph, you can
[3:20:38] easily see that if we have our X value of one and if you remember the XI or the means value was three, 1 minus 3 equals a negative two and 2 minus 3 equals a
[3:20:50] negative one, so on and so forth. And we can easily fill in the column of X minus can easily fill in the column of X minus XI, Y minus YI, and then from those we can compute X minus XI squared and X minus XI times Y minus YI. And you can
[3:21:05] ahead and sum the different columns for the answers we need. So, we get a total of 10 for our X minus XI squared and a total of two for X minus XI times Y minus YI. And we plug those in, we get 2/10, which equals 0.2. So, now we know
[3:21:22] the slope of our line equals 0.2. So, we can calculate the value of C. That'd be the next step is we need to know where crosses the Y axis. And if you remember I mentioned earlier that the linear regression line has to pass through the
[3:21:36] means value, the one that we showed earlier. We can just flip back up there to that graph and you can see right here, there's our means value, which is here, there's our means value, which is three x = 3 and y = 2.8. And since we
[3:21:48] know that value, we can simply plug that into our formula y = 0.2x + c. So, we into our formula y = 0.2x + c. So, we plug that in and we get 2.8 = 0.2 * 3 + c and you can just solve for c. So, now we know that our coefficient = 2.2. And
[3:22:05] once we have all that, we can go ahead and plot our regression line y = 0.2 * x + 2.2. And then from this equation, we can compute new values. So, let's
[3:22:17] predict the values of y using x = 1 2 3 4 5 and plot the points. Remember the 1 2 3 4 5 was our original x values. So, now we're going to see what y thinks they are, not what they actually are. And when we plug those in, we get y of
[3:22:32] designated with y of p, you can see that x = 1 = 2.4, x = 2 = 2.6 and so on and so on. So, we have our y predicted values of what we think it's going to be
[3:22:44] we plot the predicted values along with the actual values, we can see the things that's very important with linear regression and any of these models is to calculate the error on all of our different values and you can see over
[3:22:59] different values and you can see over here we plotted x and y and y predict. of see what the error looks like there between the different points. So, our goal is to reduce this error. We want to minimize that error value on our linear
[3:23:12] regression model. Minimizing the distance. There are lots of ways to minimize the distance between the line and the data points, like sum of squared errors, sum of absolute errors, root mean square error, etc. We keep moving
[3:23:26] this line through the data points to make sure the best fit line has the least square distance between the data points and the regression line. So, to recap with the very simple linear regression model, we first figure out
[3:23:38] middle and then we slowly adjust the line to minimize the error. Keep in mind this is a very simple formula. The math gets even though the math is very much the same, it gets much more complex as we add in different dimensions. So, this
[3:23:52] we add in different dimensions. So, this is only two dimensions, y = mx + c, but is only two dimensions, y = mx + c, but you can take that out to x, z, i, j, q, they can plot a linear regression model on all of those using the different
[3:24:04] formulas to minimize the error. Let's go ahead and take a look at decision trees, a very different way to solve problems than the linear regression model. Decision tree is a tree-shaped algorithm used to determine
[3:24:16] a course of action. Each branch of a tree represents a possible decision, occurrence, or reaction. We have data which tells us if it is a good day to play golf. And if we were to open this data up in a general spreadsheet, you
[3:24:30] can see we have the outlook, whether it's a rainy, overcast, sunny, it's a rainy, overcast, sunny, temperature, hot, mild, cool, humidity, windy, and did I like to play golf that day, yes or no. So, we're taking a
[3:24:42] census and certainly I wouldn't want a computer telling me when I should go play golf or not, but you could imagine if you got up in the night before you're and says tomorrow would be a good day for golf for you in the morning and not
[3:24:55] a good day in the afternoon or something like that. It becomes very beneficial coming out now where it gives you suggestions and lets you know what would be fit the match for you for the next day or the next purchase or the next uh
[3:25:09] whatever, you know, next meal out. In this case, is tomorrow a good day for playing golf based on the weather coming in. And so, we come up and let's determine if you should play golf when the day is sunny and windy. So, we found
[3:25:22] sunny and windy. And suppose we draw our tree like this. We're going to have our humidity, and then we have our normal, which is if it's if you have a normal humidity, you're going to go play golf.
[3:25:34] we look at the outlook. And if the outlook is sunny, overcast, or rainy, it's going to change what you choose to do. So, if you know that it's a very high humidity and it's sunny, you're probably not going to play golf cuz
[3:25:48] fighting off the mosquitoes that are out joining you to play golf with you. Maybe to play in the rain. But if it's slightly overcast and you get just the right shadow, that's a good day to play golf and be outside out on the green.
[3:26:02] Now, in this example, you can probably make your own tree pretty easily cuz But the question is, how do you know data? What if this is much more complicated data where it's not
[3:26:15] something that you would particularly understand, like studying cancer. They take about 36 measurements of the cancerous cells, and then each one of those measurements represents somehow bulbous it is, how extended it is, how
[3:26:29] sharp the edges are. Something that as a human we would have no understanding of. So, how do we decide how to split that data up? And is that the right decision going to come up. Is this the right decision tree? For that, we should
[3:26:42] decision tree? For that, we should calculate entropy and information gain. Two important vocabulary words there are the entropy and the information gain. Entropy. Entropy is a measure of randomness or impurity in the data set.
[3:26:56] Entropy should be low. So, we want the chaos to be as low as possible. We don't want to look at it and be confused by the images or what's going on there with mixed data. And the information gain, it is a measure of decrease in entropy
[3:27:09] after the data set is split, also known as entropy reduction. Information gain information that we get out of the split to be as high as possible. Let's take a look at entropy from the mathematical side. In this case, we're going to
[3:27:25] side. In this case, we're going to denote entropy as I of P of N N, where P is the probability that you're going to play a game of golf, and N is the play the game of golf. Now, you don't really have to memorize these formulas.
[3:27:39] depending on what you're working with. But, it's important to note that this is when you see it, you're not lost when you're running your programming, unless you're building your own decision tree code in the back. And we simply have a
[3:27:53] log squared of P over P plus N minus N over P plus N times the log squared of N of P plus N. But, let's break that down and see what it actually looks like when we're computing that from the computer script side. Entropy of a target class
[3:28:07] of the data set is the whole entropy. So, we have entropy play golf. And when we look at this, if we go back to the data, you can simply count how many yeses and no in our complete data set for playing golf days. In our complete
[3:28:21] set, we find we have five days we did play golf and nine days we did not play golf. And so, our I equals, if you add those together, 9 + 5 is 14. And so, our those together, 9 + 5 is 14. And so, our I equals 5 over 14 and 9 over 14. That's
[3:28:35] our P and N values that we plug into that formula. And you can go with the 5 over 14 equals .36, 9 over 14 equals .64. And when you do the whole equation, .64. And when you do the whole equation, you get the minus .36 log squared of .36
[3:28:50] you get the minus .36 log squared of .36 minus .64 log square root of .64. And we minus .64 log square root of .64. And we get a set value. We get .94. So, we now have a full entropy value for the whole set of data that we're working
[3:29:02] with. And we want to make that entropy go down. And just like we calculated the entropy out for the whole set, we can also calculate entropy for playing golf and the outlook. Is it going to be overcast or rainy or sunny? And so, we
[3:29:15] look at the entropy we have P of sunny times E of three of two. And that just comes out how many sunny days yes and how many sunny days no over the total, which is five. Don't forget to put the We'll divide that five out later on.
[3:29:29] We'll divide that five out later on. Equals P overcast equals 4,0 plus rainy equals 2,3. And then when you do the whole setup, we have 5 over 14. Remember I said there was a total of five. 5 over 14 times the I of three of two plus 4
[3:29:44] 14 times the I of three of two plus 4 over 14 times the 4,0 and 5 14 over I of 2 3. And so, we can now compute the entropy of just the part that has to do with the forecast. And we get 0.693. Similarly, we can calculate the entropy
[3:30:00] of other predictors like temperature, humidity, and wind. And so, we look at the gain outlook. How much are we going to gain from this? Entropy play golf minus entropy play golf outlook. And we can take the original 0.94 for the whole
[3:30:13] can take the original 0.94 for the whole set minus the entropy of just the rainy day and temperature, and we end up with a gain of 0.247. So, this is our information gain. Remember we defined entropy and we
[3:30:25] defined an information gain. The higher the information gain, the lower the entropy, the better. The information gain other three attributes can be calculated in the same way. So, we have our gain for temperature equals 0.029.
[3:30:39] We have our gain for humidity equals 0.152 and our gain for a windy day equals 0.048. And if you do a quick comparison, you'll see the 0.247 is the greatest gain of information, so
[3:30:52] that's the split we want. Now, let's build a decision tree. So, we have the outlook, is it going to be sunny, overcast, or rainy? That's our first information gain. And we can continue to go down the tree using the different
[3:31:04] information gains with the largest information we can continue down the attribute with the largest information gain as the root node, and then continue to split each sub node with the largest information gain that we can compute.
[3:31:19] tongue twister to say all that, you can see that it's a very easy to view visual model. We have our outlook, we split it three different directions. If the play. And then we can split those further down if we want. So, if the
[3:31:34] outlook is sunny, but then it's also windy, if it's windy, we're not going to play. If it's not windy, we'll play. So, we can easily build a nice decision tree tomorrow and give us a nice recommendation for the day. So, we want
[3:31:50] when it's sunny and windy. Remember the original question that came out? windy. You can see by going down the tree, we go outlook sunny, outlook tomorrow. So, our little smart watch pops up and says, "I'm sorry, tomorrow's
[3:32:05] not a good day for golf. It's going to be sunny and windy." And if you're a huge golf fan, you might go, "Uh-oh, it's not a good day to play golf. We can So, we'll sit in front of the TV instead of being out playing golf in the wind.
[3:32:19] let's look at the third one of our algorithms we're investigating, support vector machine. Support vector machine is a widely used classification algorithm. The idea of support vector machine is simple. The algorithm creates
[3:32:33] a separation line which divides the classes in the best possible manner. For example, dog or cat, disease or no disease. Suppose we have a labeled sample data which tells height and weight of males and females.
[3:32:47] know whether it's going to be a male or a female. So, we start by drawing a line. We draw decision lines, but if we consider decision line one, then we will classify the individual as a male. And if we consider decision line two, then
[3:33:02] person kind of lies in the middle of the two groups, so it's a little confusing should be under. We need to know which line divides the classes correctly, but how? The goal is to choose a hyperplane and that is one of the key words they
[3:33:16] use when we talk about support vector machines. Choose a hyperplane with the greatest possible margin between the decision line and the nearest point within the training set. So, you can see here we have our support vector, we have
[3:33:30] the two nearest points to it and we draw a line between those two points. And the distance margin is the distance between the hyperplane and the nearest data point from either set. So, we actually have a value and it should be equally
[3:33:43] distant between the two points that we're comparing it to. When we draw the hyperplanes, we observe that line one has a maximum distance. So, we observe that line one has a maximum distance margin, so we'll classify the new data
[3:33:56] point correctly. And our result on this one is going to be that the new data point is male. One of the reasons we call it a hyperplane versus a line is that a lot of times we're not looking at just weight and height. We might be
[3:34:10] looking at 36 different features or dimensions. And so, when we cut it with a hyperplane, it's more of a three-dimensional cut in the data, multi-dimensional. It cuts the data a certain way and each plane continues to
[3:34:23] cut it down until we get the best fit or match. Let's understand this with the You always start with a problem statement when you're going to put some coding now. Classifying muffin and
[3:34:35] cupcake recipes using support vector machines. So, the cupcake versus the muffin. Let's have a look at our data set and we have the different recipes here. We have a muffin recipe that has uh so much flour. I'm not sure what
[3:34:48] measurement 55 is in, but it has 55. Maybe it's ounces? But uh it has a certain amount of flour, a certain amount of milk, sugar, butter, egg, baking powder, vanilla, and salt. And so, based on these measurements, we
[3:35:02] muffin or a cupcake. And you can see in this one, we don't have just two features. We don't just have height and weight as we did before between the male and female. And here, we have a number of features. In fact, in this, we're
[3:35:15] looking at eight different features to guess whether it's a muffin or a What's the difference between a muffin and a cupcake? Turns out muffins have more flour, while cupcakes have more butter and sugar. So, basically, the
[3:35:29] dessert, where the muffin's a little bit more of a fancy bread. But how do we do that in Python? How do we code that to go through recipes and figure out what the recipe is? And I really just want to say, "Cupcakes versus muffins!" Like
[3:35:44] some big professional wrestling thing. Before we start in our cupcakes versus muffins, we are going to be working in Python. There's many versions of Python, many different editors. That is one of the strengths and weaknesses of Python.
[3:35:58] Is it just has so much stuff attached to it. It's one of the more popular data science programming packages you can use. In this case, we're going to go ahead and use Anaconda and Jupyter Notebook. The Anaconda Navigator has all
[3:36:12] kinds of fun tools. Once you're into the Anaconda Navigator, you can change environments. I actually have a number of environments on here. We'll be using Python 3.6 environment. So, this is in Python version 3.6. Although, it doesn't
[3:36:27] matter too much which version you use. I usually try to stay with the 3.x cuz project that's very specifically in version 2.x. 2.7, I think, is usually what most people use in the version two. And then once we're in our Jupiter
[3:36:41] Notebook editor, I can go up and create a new file. And we'll just jump in here. In this case we're doing SVM muffin versus cupcake. And then let's start versus cupcake. And then let's start with our packages for data analysis.
[3:36:57] We almost always use a couple There's a few very standard packages we use. few very standard packages we use. We use import Oops, import import Numpy That's for number Python. They usually
[3:37:11] denote it as NP. That's very common. That's very common. And then we're going to import Pandas as PD. There's a lot of cool things you can do with the Numpy setup as far as
[3:37:26] multiplying all the values in an array in a Numpy array data array. Pandas I can't remember if we're using it actually in this data set. I think we do as an import. It makes a nice data frame. And the difference between a data
[3:37:38] frame and a Numpy array is that a data frame is more like your Excel spreadsheet. You have columns, you have indexes. So you have different ways of referencing it, easily viewing it. And there's additional features you can run
[3:37:51] on a data frame. And Pandas kind of sits on Numpy. So they You need them both in And then finally we're working with the support vector machine. So from SKLearn,
[3:38:03] we're going to use the SKLearn model. Import SVM, support vector machine. as a data scientist, you should always try to visualize your data. Some data
[3:38:17] obviously is too complicated or doesn't make any sense to the human. But if it's look at it so that you can actually see what you're doing. And for that we're going to use two packages. We're going to import matplotlib.pyplot
[3:38:30] as plt. Again, very common. And we're going to import seaborn as sns. And we'll go ahead and set the font scale in the sns right in our import line. That's what this
[3:38:42] line. That's what this semicolon followed by a line of data. We're going to set the sns. And these are great cuz the seaborn sits on top of matplotlib library, just like pandas sits on numpy. So, it adds a lot more
[3:38:55] obviously not going to get into matplotlib library and seaborn, that'd be its own tutorial. We're really just focusing on the SVM, the support vector machine from scikit-learn. And since we're in Jupyter Notebook,
[3:39:10] we have to add a special line in here for our matplotlib library. And that's your percentage sign or ampersand matplotlib library inline. Now, if you're doing this in just a straight code project, a lot of times I
[3:39:25] and I'll write it from there. You don't it'll just pop up as its own window on your computer, depending on how your computer set up. Because we're running this in the Jupyter Notebook as a
[3:39:38] browser setup, this tells it to display all of our graphics right below on the page. So, that's what that line is for. I remember the first time I ran this, I didn't know that and I had to go look that up years ago. That was quite a
[3:39:51] headache. So, matplotlib library inline is just because we're running this on the web setup. And we can go ahead and run this, make sure all our modules are in. They're all in imported, which is great. If you don't have them imported,
[3:40:03] pip or however you do it. There's a lot of other install packages out there, have to make sure these are all installed on your Python setup. The next step, of course, is we got to look at the data. Can't run a model for
[3:40:18] predicting data if you don't have actual data. So, to do that, let me go ahead and open this up and take a look. And we have our cupcakes versus muffins. And it's a CSV file or .csv meaning that it's comma separated variable.
[3:40:32] And it's going to open it up in a nice, uh, spreadsheet for me. And you can see muffin, muffin, muffin, cupcake, cupcake, cupcake. And then it's broken up into flour, milk, sugar, butter, egg, baking powder, vanilla, and salt.
[3:40:47] So, what we can do is we can go ahead and look at this data also in our and look at this data also in our Python. We're going to use our pandas module .read
[3:41:01] CSV. Remember it's a comma separated variable. And the file name happened to be cupcakes versus muffins. Oops, I got cupcakes versus muffins. Oops, I got double brackets there.
[3:41:22] Because the program I loaded or the the place I saved this particular Python program is in the same folder, we can get by with just the file name. But different location, you have to also put down the full path on there.
[3:41:40] actually In line you can do this, but let me do the full print. And you can let me do the full print. And you can just type in recipes.head in the Jupyter notebook. But if you're running in code in a different script,
[3:41:53] you'd need to go ahead and type out the whole print recipes.head. And pandas knows this is going to do the first five lines of data. And if we flip back on over to the spreadsheet where we opened up our CSV file,
[3:42:07] uh, you can see where it starts on line two. This one calls it zero. And then 2 3 4 5 6 is going to match. Go and close that up cuz we don't need that anymore. And it always starts at zero. And these are know, indexes it since we
[3:42:21] didn't tell it to use an index in here. So, that's the index number for the left-hand side. And it automatically took the top row as uh labels. So, Pandas using it to read a CSV is just really
[3:42:35] slick and fast. One of the reasons we love our Pandas. Not just because love our Pandas. Not just because they're cute and cuddly teddy bears. they're cute and cuddly teddy bears. And let's go ahead and plot our data.
[3:42:47] I'm not going to plot all of it. I'm just going to plot the uh sugar and Now, obviously you can see where they get really complicated if we have tons of different features. And so, you'll break them up
[3:43:00] and maybe look at just two of them at a time to see how they connect. ahead and use Seaborn. So, that's our SNS. And the command for that is sns.lmplot. And then, the two different variables
[3:43:15] And then, the two different variables I'm going to plot is flour and sugar. Data equals recipes. The hue equals type. And this is a lot of fun because it knows that this is Pandas coming in. Um so, this is one of
[3:43:28] the powerful things about Pandas mixed with Seaborn and doing And then, we're going to use a palette set one. There's a lot of different sets Seaborn. Or do a regular uh fit regular equals
[3:43:44] false. So, we're not really trying to fit anything. And it's a scatter KWS. in Seaborn. Half of these you could probably leave off when you run them. that these were the best settings for doing a Seaborn plot.
[3:43:59] And let's go ahead and run that. And because it does it in line, it just puts it right on the page. And you can see right here that just based on sugar and flour alone, there's a definite split.
[3:44:14] And we use these models because you can actually look at it and say, "Hey, if I drew a line right between the middle of the blue dots and the red dots, we'd be able to do an SVM and and a hyperplane right there in the middle."
[3:44:35] process our data. And we're going to break that up into two parts.
[3:44:47] muffin or a cupcake. Well, a computer doesn't know muffin or cupcake. It knows So, what we're going to do is we're going to create a type label. And from this, we'll create a NumPy
[3:44:59] np.where And this is where we can do some logic. We take our recipes from our panda and where our type equals muffin, it's going to be zero. And then, if it doesn't equal muffin, which is cupcakes,
[3:45:14] it's going to be one. So, we create our type label. our training model, remember we have to have a training data. This is what we're going to train it with is that it's zero or one. It's a muffin or it's not.
[3:45:29] recipe features. And if you remember correctly from right up here, the first column is type. So, we really don't need the type column cuz that's our muffin or cupcake. And in
[3:45:42] cuz that's our muffin or cupcake. And in pandas, we can easily sort that out. We take our value recipes .columns. That's a pandas function built into pandas.
[3:45:56] .values. Converting them to values. So, it's just the column titles going across the top. And we don't want the first one. So, what we do is since it always one. So, what we do is since it always starts at zero, we want one
[3:46:08] colon till the end. And then, we want to go ahead and make this a list and this converts it to a list of and this converts it to a list of strings.
[3:46:21] look and see what we're looking at for the features. Make sure it looks right. the features. Make sure it looks right. We go ahead and run that. run that. And we can see we have flour, milk, sugar, butter, egg, baking powder,
[3:46:39] vanilla, and salt. And that matches what we have up here when we print it out. Everything but the type. So, we have our features and we have our label. Now, the recipe features is just the titles of the columns and we actually
[3:46:53] need the ingredients. options. One, we could run it over all the ingredients.
[3:47:05] And when you're doing this usually you do, but for our example we want to limit it so you can easily see what's going on. Cuz if we did all the ingredients we have you know, that's what um seven, eight different hyperplanes that
[3:47:18] would be built into it. We only want to look at one so you can see what the SVM is doing. And so, we'll take our recipes and we'll do just flour and sugar. Again, you can replace that with your
[3:47:30] recipe features and do all of them, but we're going to do just flour and sugar values. We don't need to make a list out of it cuz it's not string values. These are actual values on there. And we can go ahead and
[3:47:43] just print ingredients and you can see what that Uh and so, we have just the amount of flour and sugar. Just the two sets of plots.
[3:47:57] take this over here and take our recipe features. And so, if we decided to use all the recipe features of different data. So, it just strips out all the labels and everything. We
[3:48:11] just have just the values. But, because we want to be able to view this easily in a plot later on, we'll go ahead and take that and just do flour and sugar.
[3:48:24] And we'll run that and you'll see it's just the two columns. our model. We'll go ahead and just call it model. And it's a SVM. We're using a package called SVC.
[3:48:42] and set the kernel equals linear. So, it's using a specific setup on there. And if we go to the reference on their website for the SVM, you'll see that there's about There's eight of them here. Three of them are
[3:48:55] for regression, three are for classification. The SVC, support vector classification, is probably one of the most commonly used. And then there's also one for detecting outliers, and another one that has to do
[3:49:09] with something a little more specific on the model. But, SVC and SVR are the two most commonly used, standing for support vector classifier and support vector regression. Remember, regression is an actual value, a float value or whatever
[3:49:24] you're trying to work on. And SVC is a classifier, so it's a yes, no, true, false. But, for this, we want to know 0 1, muffin cupcake. We go ahead and create our model. And
[3:49:36] once we have our model created, we're going to do model.fit. And this is very their models are followed with the fit command. And what we put into the fit, what we're training with it, is we're putting in
[3:49:49] the ingredients, which in this case we limited to just flour and sugar, and the type label, is it a muffin or cupcake? Now, in more complicated data science series, you'd want to split into We
[3:50:03] won't get into that today. We split it into a training data and test data. And it into thirds, where a third is used for where you switch between which one's things go into that. It gets very complicated when you get to the higher
[3:50:18] end. Not overly complicated, just an extra step, which we're not going to do data. And let's go ahead and run this. And now we have our model fit. And uh I got a error here, so let me fix
[3:50:31] that real quick. It's capital SVC, it turns out. I did it lowercase. Support vector classifier. There we go. Let's go ahead and run that.
[3:50:44] information that it prints out automatically. These are the defaults of You notice that we changed the kernel to linear. And there's our kernel linear on the printout. And there's other different settings you can mess with.
[3:50:59] for right now. For this, we don't really need to mess with any of those. So, next, we're going to dig a little bit into our newly trained model.
[3:51:12] show you on a graph. And let's go ahead and get the separating And we're going to say uh we're going to use a W for our variable on here.
[3:51:26] And we're going to do model.coefficient_zero. digging into the model. So, we've already got a prediction and a train. This is the math behind it that we're looking at right now.
[3:51:41] And so, the W is going to represent two different coefficients. And if you remember, we had Y = MX + C. So, these coefficients are connected to that, but in two-dimensional. It's a
[3:51:55] plane. We don't want to spend too much time on this because you can get lost in the confusion of the math. So, if you're a math wiz, this is great. You can go through here and you'll see that we have
[3:52:07] through here and you'll see that we have a equals minus w zero over w of one. there. And that's basically the slope that we're generating. And then we're going to do build an xx.
[3:52:21] What is xx? We're going to set it up to a numpy array. There's our np.linspace. So, we're creating a line of values between 30 and 60. So, it just
[3:52:33] creates a set of numbers for x. And then if you remember correctly, we have our formula y equals the slope times x plus the intercept. Well, to make this work, we can do this as yy
[3:52:49] work, we can do this as yy equals the slope times each value in that array. That's the neat thing about numpy. So, when I do a times xx, which is a whole numpy array of values, it multiplies a across all of them.
[3:53:03] And then it takes those same values and we subtract the model intercept. That's your uh what we had mx plus c. So, that'd be the what we had mx plus c. So, that'd be the c from the formula y equals mx plus c.
[3:53:15] from. A little bit confusing cuz it's digging out of these different arrays. going to take this and we're going to go ahead and plot it. So, plot the parallels to separating hyperplane that pass through the support
[3:53:29] vectors. And so, we're going to create b equals a model support vectors. Pull on our support vectors out there. Here's our yy, which we now know is a set of data. And we have we're going to create yy down equals a times xx plus
[3:53:46] b1 minus a times b0. And then model support vector b is going to be set that to a new value the minus one set up and YY up equals A times XX
[3:53:58] one set up and YY up equals A times XX plus B1 minus A times B0. load these variables up. If you want to know a understand a little bit more of know a understand a little bit more of what's going on, you can see if we print
[3:54:15] see it's an array. It's this is a line. It's going to have in this case between 30 and 60, so it's going to be 30 variables in here. variables in here. And the same thing with YY up, YY down.
[3:54:27] minute on a graph so you can see what those look like. here and run that so it loads up the variables. Nice clean slate. I'm just going to copy this from before.
[3:54:40] Remember this our SNS dot our seaborn plot LM plot flower sugar? what that looks like. It's just a straight graph on there.
[3:54:52] And then one of the new things is because seaborn sits on top of pi plot, we can do the pi plot for the line going through. And that is simply plt.plot.
[3:55:07] and YY are two corresponding values XY. And then somebody played with this to figure out that the line width equals two and the color black would look nice. thing with the pi plot on there. And you can see when we do this, it's
[3:55:22] And you can see when we do this, it's just doing flower and sugar on here. Corresponding line between the sugar and the flower and the muffin versus the flower and the muffin versus cupcake.
[3:55:35] Um and then we generated the um support vectors the YY down and YY up. So let's like. So we'll do our PL plot. So we'll do our PL plot. And again this is all against XX the our
[3:55:48] And again this is all against XX the our X value. But this time we have YY And let's do something a little fun with this. this. We can put in a K- -
[3:56:00] line. And if we're going to do the down one, we also want to do the up one. So here's our YY
[3:56:12] up. And when we run that, it adds both sets And so here's our support, and this is what you expect. You expect these two points. So the dash lines go through the nearest muffin and the nearest cupcake
[3:56:27] when it's plotting it. And then your SVM goes right down the middle. So it gives it a nice split in our data, and you can see how easy it is to see based just on sugar and flour which one's a muffin or a cupcake.
[3:56:40] which one's a muffin or a cupcake. Let's go ahead and create a function to predict muffin or cupcake. I've got my recipes I've pulled off the um
[3:56:56] difference between a muffin or a cupcake. And so we need a function to push that through. And uh we'll create a function with def. And let's call it muffin or cupcake. And remember, we're just doing flour and
[3:57:10] ingredients. And that actually is a pretty good split. You really don't need and sugar. And let's go ahead and do an if else And let's go ahead and do an if else statement. So if model predict
[3:57:28] so we take our model and we do it run a predict. That's very common in SK learn where you have a dot predict. You put the data in, and it's going to return a value. In this case, if it equals zero, then print you're you're at a muffin
[3:57:40] recipe. Else, if it's not zero, that means it's recipe. That's pretty straightforward for function or def for definition. d e f is how you do that Python.
[3:57:55] function you should run something in it and so let's run a cupcake and we're going to send it values 50 and 20. A muffin or cupcake, I don't know what it gives us. And it says, "Oh, it's a muffin. You're
[3:58:08] looking at a muffin recipe." So it very easily predicts whether we're looking at a muffin or a cupcake recipe. Let's plot this. There we go. Plot this actually looks like. And I'm just going to copy and paste it
[3:58:23] from below where we're plotting all the points in there. before. If I run it, you'll see it has all the points and the And what we want to do is we want to add another point.
[3:58:36] another point. And we'll do plt.plot. for our test we did 50 and 20. And then somebody went in here and decided we'll do a yo for yellow or it's
[3:58:50] kind of a orangish yellow color is going to come out. Marker size nine. Those are else played with them to come up with the right setup so it looks good. And you can see there it is graphed. Clearly a muffin.
[3:59:04] In this case, in cupcakes versus muffins, the muffin has won. And if you'd like to do your own muffin cupcake contender series, you certainly can send a note down below and the team at Simplilearn will send you over the
[3:59:20] cupcake. And that's true of any of the data. We didn't actually run a plot on it earlier. We had men versus women. You can also request that information to run it on your data setup so you can test that out.
[3:59:34] So to go back over our setup, we went ahead for our support vector machine code. We did a predict 40 parts flour, 20 parts sugar. I think it was different muffin or a cupcake. Hence, we have built a classifier using SVM, which is
[3:59:50] able to classify if a recipe is of a cupcake or a muffin. What's in it for you? We're going to cover clustering. What is clustering? K-means clustering, which is one of the most common used clustering tools out there, including a
[4:00:05] flowchart to understand K-means clustering and how it functions. And then we'll do an actual Python live demo on clustering of cars based on brands. Then we're going to cover logistic regression. What is logistic regression?
[4:00:18] Logistic regression curve and sigmoid function. And then we'll do another Python code demo to classify a tumor as malignant or benign based on features. And let's start with clustering. Suppose we have a pile of books of different
[4:00:32] genres. Now, we divide them into different groups like fiction, horror, education. And as we can see from this young lady, she definitely is into heavy and the maple Canadian leaf on her
[4:00:46] education. And we want to go ahead and divide our books up. Well, organizing objects into groups based on similarity is clustering. And in this case, as talking about clustering things with no one categories. But you can also use it
[4:01:02] the categories. You just know that you conquer the data and to organize it better. But in this case, we're going to categories. And let's just take a deeper look at that. We're going to use K-means
[4:01:17] clustering. K-means clustering is probably the most commonly used clustering tool in the machine learning library. K-means clustering is an example of unsupervised learning. If you remember from our previous thing, it is
[4:01:30] used when you have unlabeled data. So, we don't know the answer yet. We have a bunch of data that we want to cluster to different groups. To find clusters in the data based on feature similarity. So, we've introduced a couple terms
[4:01:43] here. We've already talked about unsupervised learning and unlabeled data. So, we don't know the answer yet. We're just going to group stuff together and see if we can find an answer of how things connect. We've also introduced
[4:01:55] feature similarity. Features being different features of the data. Now, with books, we can easily see fiction and horror and history books, but a lot of times with data, some of that information isn't so easy to see, right,
[4:02:09] K-means is one of those tools where we can start finding things that connect, that match with each other. Suppose we have these data points and want to assign them into a cluster. Now, when I look at these data points, I would
[4:02:22] just by looking at them. I'd say two of these group of data kind of come together. But in K-means, we pick K clusters and assign random centroids to clusters, where the K clusters represents two different clusters. We
[4:02:37] centroids to the clusters, then we compute distance from objects to the centroids. Now, we form new clusters based on minimum distances and calculate the centroids. So, we figure out what the best distance is for the centroid,
[4:02:52] then we move the centroid and recalculate those distances. Repeat previous two steps iteratively till the cluster centroids stop changing their positions and become static. Repeat previous two steps iteratively till the
[4:03:05] positions become static. Once the clusters become static, then K-means clustering algorithm is said to be converged. And there's another term we see throughout machine learning is converged. That means whatever math
[4:03:18] we're using to figure out the answer has come to a solution or it's converged on an answer. Shall we see the flowchart to understand? Make a little bit more sense by putting it into a a nice easy step-by-step. So, we start, we choose K,
[4:03:31] we'll look at the elbow method in just a moment. We assign random centroids to clusters, and sometimes you pick the centroids because you might look at the data in a in a graph and say, "Oh, these are probably the central points." Then
[4:03:44] we compute the distance from the objects to the centroids. We take that and we form new clusters based on minimum distance and calculate their centroids. Then we compute the distance from objects to the new centroids. And then
[4:03:57] we go back and repeat those last two steps. We calculate the distances. So, as we're doing it, it brings into the new centroid, and then we move the centroid around and we figure out what the best which objects are closest to
[4:04:09] switch from one centroid to the other as the centroids are moved around. And we continue that until it is converged. Let's see an example of this. Suppose we have this data set of seven individuals and their score on two topics, A and B.
[4:04:25] So, here's our subject, in this case referring to the person taking the test. And then we have subject A where we see what they've scored on their first can see what they score on the second subject. Now, let's take two farthest
[4:04:40] apart points as initial cluster centroids. Remember, we talked about selecting them randomly or we can also just put them in different points and together. Either one works okay, depending on what kind of data you're
[4:04:53] working on and what you know about it. So, we took the two furthest points, one and one and five and seven. And now let's take the two furthest apart points as initial cluster centroids. Each point is then assigned to the closest cluster
[4:05:07] with respect to the distance from the centroids. So, we take each one of these distance, and you can see that if we measured each of those distances, and you'd use it the Pythagorean theorem for a triangle in this case because you know
[4:05:21] the X and the Y and you can figure out the diagonal line from that or you just That'd be kind of silly, but it would work if you're just eyeballing it. You can see how they naturally come together in certain areas. Now we again calculate
[4:05:35] the centroids of each cluster. So cluster one and then cluster two and we look at each individual dot. There's one, two, three were in one cluster. The centroid then moves over. It becomes 1.8 {comma} 2.3. So remember it was at one
[4:05:50] and one? Well, the very center of the data we're looking at would put it at the 1. roughly 2 2, but 1.8 and 2.3. And the second one, if we wanted to make the of all the different distances to that centroid, we come up with 4 {comma} 1
[4:06:06] and 5 4. So we've now moved the centroids. We compare each individual's distance to its own cluster mean and to that of the opposite cluster and we find and build a nice chart on here that the as we move that centroid around, we now
[4:06:20] have a new different kind of clustering of groups and using Euclidean distance between the points and the mean, we get the same formula you see as new formulas dots, distance to the mean centroid of the cluster and distance to the mean
[4:06:33] Only individual three is nearer to the mean of the opposite cluster, cluster two, than its own cluster one. And you can see here in the diagram where we've So when we've moved the cluster the centroids of the clusters over, one of
[4:06:48] the points shifted to the other cluster because it's closer to that group of individuals. Thus, individual three is relocated to cluster two resulting in a new partition and we regenerate all those numbers of how close they are to
[4:07:01] the different clusters. For the new clusters, we will find the actual cluster centroids. So now we move the centroids over and you can see that clusters on here. On comparing the
[4:07:13] distance of each individual's distance to its own cluster mean and to that of the opposite cluster, we find that the data points are stable. Hence, we have remember, I brought up a concept earlier, K-means on the K-means
[4:07:26] algorithm. Choosing the right value of K will help in less number of iterations. And to find the appropriate number of clusters in a data set, we use the elbow clusters in a data set, we use the elbow method. And within sum of squares, WSS
[4:07:39] is defined as the sum of the squared distance between each member of the cluster and its centroid. And so, you see what we've done here is we have the number of clusters. And as you do the same K-means algorithm over the
[4:07:53] what that centroid looks like, and you find the optimal, you can actually find the optimal number of clusters using the elbow. The graph is called as the elbow method. And on this, we guessed at two just by looking at the data. But as you
[4:08:06] can see, the slope You actually just look for right there where the elbow is answer that we want two different to start with K-means equals two. A lot of times, people end up computing K-means equals two, three, four, five until they
[4:08:20] find the value which fits on the elbow joint. Sometimes, you can just look at the data, and if you're really good with that specific domain Number domain, I that that where to pick those numbers or where to start guessing at what that K
[4:08:34] value is. So, let's take this. And we're going to use a use case using K-means clustering to cluster cars into brands using parameters such as horsepower, cubic inches, make, year, etc. So, we're going to use the data set cars data
[4:08:50] having information about three brands of cars, Toyota, Honda, and Nissan. And we'll go back to my favorite tool, the Anaconda Navigator with the Jupyter Notebook. And let's go ahead and flip over to our Jupyter Notebook. And in our
[4:09:05] Jupyter Notebook, I'm going to go ahead and just paste the basic code that we We're not going to go too much into this code because we've already discussed NumPy, we've already discussed Matplotlib Library and Pandas. NumPy
[4:09:18] being the number array and Pandas being the Pandas data frame and Matplotlib for the graphing. And don't forget since if you're using the Jupyter Notebook, you do need the Matplotlib Library inline so that it plots everything on the screen.
[4:09:30] If you're using a different Python editor, then you probably don't need that cuz they'll have a pop-up window on your computer. And we'll go ahead and run this just to load our libraries and our setup into here. The next step is of
[4:09:43] course to look at our data, which I've already opened up in a spreadsheet. And you can see here we have the miles per gallon, cylinders, cubic inches, horsepower, weight pounds, how you know how heavy it is, time it takes to get to
[4:09:56] 60. My car is probably on this one at about 80 or 90. What year it is. So this is you can actually see this is kind of older cars and then the brand, Toyota, Honda, Nissan. So the different cars are coming from all the way from 1971 if we
[4:10:11] scroll down to the '80s. So we have between the '70s and '80s the number of cars that they've put out. And let's we come back here, we're going to do importing the data. So we'll go ahead and do data set equals and we'll use
[4:10:24] Pandas to read this in and it's from a CSV file. Remember you can always post this in the comments and request the data files for these, either in the comments here on the YouTube video or go to simplylearn.com and request that. The
[4:10:38] cars.csv I put it in the same folder as the code that I've stored. So my Python code is stored in the same folder so I don't have to put the full path. If you have to change this and double-check your name variables. And we'll go ahead
[4:10:52] and run this and we've chosen data set arbitrarily cuz you know it's a data set we're importing. And we've now imported our cars.csv into the data set. So we're going to create the X data. This is the one that we're going to try
[4:11:06] to figure out what's going on with. And then there is a number of ways to do this, but we'll do it in a simple loop so you can actually see what's going on. so you can actually see what's going on. So we'll do for I in x.columns.
[4:11:18] columns and a lot of times it's important I'll I'll make lists of the columns and do this because I might remove certain columns or there might be columns that I want to be processed differently. But
[4:11:31] for this, we can go ahead and take x of I and we want to go fill in a and that's a pandas command. But the question is what are we going to fill the missing data with? We definitely don't want to just put in a number that doesn't
[4:11:45] actually mean something. And so one of the tricks you can do with this is we can take x of I and then in addition to that we want to go ahead and turn this into an integer cuz a lot of these are integers. So we'll go ahead and keep it
[4:11:58] integers and let me add the bracket here and a lot of editors will do this. bracket. Make sure you get that second bracket in there if it's a double bracket. That's always something that happens regularly. So once we have our
[4:12:10] integer of x of I, this is going to fill in any missing data with the average. And I was so busy closing one set of brackets, I forgot that the mean is also we can see here we're going to fill in all the data with the average value for
[4:12:24] that column. So if there's missing data it's in the average of the data it does have. And then once we've done that, we'll go ahead and loop through it again and just check and see to make sure everything is filled in correctly. And
[4:12:37] everything is filled in correctly. And we'll print and then we take x is null and this returns a set of the null value or the how many lines are null and we'll just sum that up to see what that looks like. And so when I run this, and so
[4:12:49] with the x, what we want to do is we want to remove the last column because trying to see if we can cluster these things and figure out the models. There is so many different ways to sort the x out. For one, we could take the X
[4:13:04] and we could go data set, our variable we're using, and use the I location, one of the features that's in Pandas. And we could take that and then take all the rows and all but the last column of the data set.
[4:13:20] And at this time we could do values. We just convert it to values. So, that's one way to do this. And if I let me just put this down here and print X. It's a capital X we chose. And I run this, you can see it's just the values. We could
[4:13:33] also take out the values and it's not going to return anything cuz there's no values connected to it. What I like to do with this is instead of doing the I location, which does integers, more common is to come in here and we have
[4:13:47] our data set and we're going to do data set dot or data set dot columns. And remember that list all the columns. So, if I come in here
[4:13:59] if I come in here let me just mark that as red and I print let me just mark that as red and I print data set dot columns. have my MPG, cylinders, everything, including the brand, which we don't
[4:14:13] want. So, the way to get rid of the brand would be to do data columns of everything but the last one, minus one. So, now if I print this, you'll see the And so, I can actually just take data
[4:14:27] set columns minus one and I'll put it right in here for the columns we're going to look at. And uh let's unmark this and unmark this. And now if I do an X dot head
[4:14:44] I now have a new data frame and you can see right here we have all the different of the year. And it turns out when you start playing with the data set, you're going to get an error later on and it'll say, "Cannot
[4:14:57] an error later on and it'll say, "Cannot convert string to float value." And things, the way they recorded them, must have been recorded as strings. So, we have a neat feature in here on Pandas to convert. And it is simply convert
[4:15:12] convert. And it is simply convert objects. Oops, convert {underscore} numeric numeric equals true. And yes, I
[4:15:26] have it memorized the convert numeric in there. If I'm working with a lot of these things, I remember them, but um depending on where I'm at, what I'm doing, I usually have to look it up. And we run that. Oops, I must have missed
[4:15:38] something in here. Let me double-check my spelling. And when I double-check my underscore in the convert objects. And when I run this, it now has everything when I run this, it now has everything converted into a numeric value, cuz
[4:15:51] that's what we're going to be working with is numeric values down here. And the next part is that we need to go through the data and eliminate null values. Most people when they're doing small amounts working with small data
[4:16:04] a null value, and they have to go back and do this. So, you know, be aware, things are going to pop up, and sometimes you go backwards to fix it.
[4:16:16] exploring the data and understanding what you have. And I should have done this earlier, but let me go ahead and increase the size of let me go ahead and increase the size of my window one notch.
[4:16:32] So, we'll do for I in working with X {dot} columns, we'll page through all {dot} columns, we'll page through all the columns, and we want to take X of I, and we're going to change that, and we're going to alter it. And so, with
[4:16:45] this, we want to go ahead and fill in X of I. Pandas has the fill NA. And that just fills in any non-existent missing data. And we'll put my brackets up, and there's a lot of different ways to fill
[4:17:00] If you have a really large data set, some people just void out that data cuz if and then look at it later in a separate exploration of data. One of the tricks we can do is we can take our column
[4:17:14] and we can find the means. And the means is in other uh quotation So, when we take the columns, we're going to fill in the the non-existing one with the means. The problem is that returns a decimal float. So, some of
[4:17:28] these aren't decimals. Certainly, maybe a little careful of doing this, but for this example, we're just going to fill it in with the integer version of this. It keeps it on par with the other data that isn't a decimal point.
[4:17:45] A lot of times you do this first part first to double-check, then you do the make sure you did it right. So, we're going to go through and test for missing data. And one of the ways you can do that is
[4:18:00] simply go in here and take our X of I column. So, it's going to go through the X of I column and says is null. So, it's going to value, it actually goes through the all the rows of each column is null, and
[4:18:15] then we want to go ahead and sum that. So, when we take that, we add the sum value. These are all pandas, so is null is a panda command, and so is sum. And if we go through that, we go ahead and run it.
[4:18:30] you'll see that all the columns have zero null values. So, we've now tested and clean. We have no null values. Everything is now a number value. We turned it into numeric. And we've removed the last column in our data.
[4:18:45] And at this point, we're actually going to start using the elbow method to find the optimal number of clusters. So, we're now actually getting into the SK learn part. Uh the K means clustering on here.
[4:18:59] one more notch so you can see what I'm typing in here. cluster I'm going to import K
[4:19:15] means. I always forget to capitalize the K and the M when I do this. So, it's capital K, capital M, K means. And we'll go ahead and create a um array WCSS
[4:19:29] equals Let me get an empty array. If you remember from the elbow method from our slide within the sums of squares, WSS is defined as the sum of squared distance between each member of the cluster and
[4:19:43] its centroid. So, we're looking at that change in differences as far as a squared distance. And we're going to run this over a number of K mean values. In fact, let's go for I in range. We'll do 11 of them.
[4:19:59] Range 0 of 11. And the first thing we're going to do is we're going to create the actual We'll do it all lower case.
[4:20:12] object from the K means we just imported. And the variable that we want to put into this is in clusters. And we're going to set that equals to I. That's the most important
[4:20:28] one cuz we're looking at how increasing the number of clusters changes our answer. There are a lot of settings to the K means. Our guys in the back did a great job just kind of playing with some of them.
[4:20:41] The most common ones that you see in a lot of stuff is how you would knit your K-means. So, we have K-means plus plus plus. This is just a tool to let the model itself be smart how it picks the
[4:20:54] centroids to start with its initial centroids. We only want to iterate no more than 300 times. We have a max iteration we put in there. We have an the n_init, the random_state equals zero. You really don't need to
[4:21:06] first learning this. As you start digging in deeper, you start finding that these are shortcuts that will speed up the process that we're working with is the n_clusters equals i. So, we're going to
[4:21:21] literally train our K-means 11 times. We're going to do this process 11 times. And if you're working with a big data, you know, the first thing you do is you run a small sample of the data so you can test all your stuff on it. And you
[4:21:35] can already see the problem that if I'm going to iterate through a terabyte of data 11 times, and then the K-means itself is iterating through the data multiple times, that's a heck of a process. So, you got to be a little
[4:21:47] careful with this. A lot of times though, you can find your elbow, using number on a sample of data, especially if you're working with larger data So, we want to go ahead and take our K-means and we're just going to fit it.
[4:22:01] learn, very common, that you fit your model. And if you remember correctly, X. And once we fit this value, we go back to the array we made, and we want to go ahead and just append that value on the
[4:22:16] And it's not the actual fit we're putting in there. It's when it generates it, it generates the value you're looking for is inertia. So, K-means.inertia will pull that specific value out that we need.
[4:22:30] And let's get a visual on this. We'll do our plt.plot, and what we're plotting here is first the X axis, which is range 0 to 11. So, that will generate a nice little plot there. And the WCSS for our Y axis.
[4:22:52] title. And let's see, we'll just give it the elbow method for the title. And let's get some labels. So, let's go ahead and x label. And we'll we'll do number of clusters
[4:23:06] And we'll we'll do number of clusters for that. And plt y label. And for that we can do oops, there we go. WCSS since that's what we're doing on the plot on there. And finally, we want to go ahead and display our graph,
[4:23:18] which is simply plt. oops. .show. There we go. And because we have it set to inline, it'll appear inline. Hopefully, I didn't make a type error on there.
[4:23:35] graph. You can see a very nice elbow joint there at two. And again, right around three and four. And then after that, there's not very much. Now, as a data scientist, if I was looking at this,
[4:23:48] actually try both of them to see what the output looked like. And they've we're just going to use three as a setup that looks like when we actually use this to show the different kinds of
[4:24:02] this to show the different kinds of cars. means to the cars data set. And basically, we're going to copy the code that we looked through up above, where K means equals K means number of clusters.
[4:24:16] And we're just going to set that number of clusters to three. for. And you could do three and four on this and graph them just to see how they curious to look at that. But for this, we're just going to set it to three. Go
[4:24:29] go and create our own variable, Y K means for our answers. And we're going to set that equal to whoops, I did a double equal there, to K But we're not going to do a fit. We're going to do a fit predict is the set up
[4:24:46] you want to use. And when you're using untrained models, you'll see a slightly then you see just the predict, but we want to both fit and predict the K means on this. And that's fit underscore predict and then our capital X is the
[4:25:01] data we're working with. And before we plot this data, we're going to do a little Pandas trick. We're going to take our X value and we're going to set X as matrix. So we're converting this into a nice
[4:25:13] rows and columns kind of set up, but we want the we're going to have columns equals none, so it's just going to be a matrix of data in here. And let's go ahead and run that. It's a little warning. You'll see these
[4:25:25] warnings pop up cuz things are always being updated. So there's like minor changes in the versions in future versions. Instead of matrix, now that it's more common to set it dot values instead of doing as matrix. But as
[4:25:38] matrix works just fine for right now and you'll want to update that later on. But let's go ahead and dive in and plot this and see what that looks like. And before we dive into plotting this data, I always like to take a look and
[4:25:50] see what I am plotting. So let's take a look at Y out down here. And we see we have an array of answers. And we see we have an array of answers. We have 2 1 0 2 1 2. So it's clustering
[4:26:03] rows of data based on the three different spaces it thinks it's going to be. And then let's go ahead and print X and see what we have for X. And we'll see that X is an array, it's a matrix. So we
[4:26:19] have our different values in the array. And what we're going to do, it's very hard to plot all the different values in the array. So we're only going to be looking at the first two or at position zero and one.
[4:26:32] And if you were doing a full presentation in front of the board you might actually do a little different than and dig a little deeper into the different columns we looked at. But, we'll only look at columns one and two
[4:26:46] So, let's go ahead and clear this data out of here and let's bring up our plot. We're going to do a scatter plot here. So, PLT scatter This looks a little complicated. So,
[4:27:01] let's explain what's going on with this. We're going to take the X values and we're only interested in Y of K means equals zero, the first cluster. And then we're going to take value zero for the X axis and then we're going to
[4:27:16] interested in K means equals zero, but we're going to take the second column. So, we're only looking at the first two columns in our answer or in the data. And then the guys in the back played
[4:27:29] pretty. And they discovered that it looks good with as a size equals 100, that's the size of the dots. We're going to use red for this one and when they were looking at the data and
[4:27:43] Toyota on this. We're just going to go ahead and label it Toyota. Again, that's something you really have to explore in here as far as playing with those go ahead and hit enter in there and I'm
[4:27:57] just going to paste in the next two lines, which is the next two cars. And this is our Nissan and Honda. And you'll see with our scatter plot we're now looking at Y underscore K means equals one.
[4:28:10] And we want the zero column and Y K means equals two. Again, we're looking at just the first two columns, zero and one. And each of these rows then one. And each of these rows then corresponds to Nissan and Honda.
[4:28:22] And I'll go ahead and hit enter on there. And uh finally, let's take a look and put the centroids on there. Again, we're going to do a scatter plot. And on the centroids, you can just pull that from our K-means, the uh model we
[4:28:36] created, dot cluster centers, and we're going to just do um all of them in the first number and all of them in the second number, which is 0 of them in the second number, which is 0 1, cuz you always start with 0 and 1.
[4:28:51] and everything to make it look good. We'll do a size of 300. We're going to them uh so it's good to have some good labels, centroids. And then we do want to do a title, PLT title.
[4:29:07] And pop up there. PLT title, cuz you always make want to make your graphs look pretty, and we'll call it clusters of car make. And one of the features of the plot library is you can add a
[4:29:21] legend. It'll automatically bring in it since we've already labeled the different aspects of the legend with Toyota, Nissan, and Honda. show. So we can actually see it. And remember,
[4:29:33] it's in line. Uh so if you're using a different editor that's not the Jupiter notebook, you'll get a pop-up of this. And you should have a nice set of clusters here. So we can look at this and we have uh clusters of Honda in
[4:29:45] and we have uh clusters of Honda in green, Toyota in red, Nissan in purple. And you can see where they put the centroids to separate them. Now, when we're looking at this, we can also plot a lot of other different data
[4:29:57] on here as far cuz we only looked at the first two columns. This is just column one and two, or 0 1 as as you label them in computer scripting. But you can see make, and we've been able to pull out the data, and you can see how just these
[4:30:11] two columns form very distinct clusters of data. might take a look and say, "Well, what makes these different?" Almost going in reverse, you start looking at the data and pulling apart the columns to find
[4:30:25] out why is the first group set up the way it is. Maybe you're doing loans and not defaulting on their loans?" And why is the last group defaulting on their loans? And why is the middle group 50% defaulting on their bank loans? And you
[4:30:39] start finding ways to manipulate the data and pull out the answers you want. K-means for clustering, let's move on to the next topic.
[4:30:53] Now, let's look into logistic regression. The logistic regression algorithm is the simplest classification algorithm used for binary or multi-classification problems. And we can see we have our little girl from
[4:31:05] Canada, who's into horror books, is back. That's actually really scary when eyes. In the previous tutorial, we learned about linear regression, dependent and independent variables. So, to brush up, y = mx + c. Very basic
[4:31:21] to brush up, y = mx + c. Very basic algebraic function of y and x. The dependent variable is the target class variable we are going to predict. variable we are going to predict. The independent variables, x1 to all the
[4:31:33] way up to xn, are the features or attributes we're going to use to predict the target class. We know what a linear regression looks like, but using the graph, we cannot divide the outcome into categories. It is really hard to
[4:31:45] categories. It is really hard to categorize 1.5, 3.6, 9.8. Uh for example, a linear regression graph can tell us that with increase in number of hours studied, the marks of a student will increase. But, it will not tell us
[4:31:59] whether the student will pass or not. In such cases, where we need the output as categorical value, we will use logistic regression. And for that, we're going to use the sigmoid function. So, you can see here we have our marks, 0 to 100,
[4:32:13] number of hours studied. That's going to be what they're comparing it to in this example. And we use the formula line that says y = mx + c. And when we use that says y = mx + c. And when we use the sigmoid function, we have p = 1 / 1
[4:32:25] the sigmoid function, we have p = 1 / 1 + e to the - y. It generates a sigmoid curve. And so, you can see right here when you take the ln, which is the natural logarithm. I always thought it should be nl, not ln. That's just the
[4:32:39] inverse of uh e, your e to the - y. And so, when we do this, we get ln of p over so, when we do this, we get ln of p over 1 - p = m * x + c. That's the sigmoid curve function we're looking for. And we can zoom in on the function, and you'll
[4:32:53] see that the function as it derives goes to 1 or to 0, depending on what your x value is. And the probability, if it's greater than 0.5, the value is automatically rounded off to 1, indicating that the student will pass.
[4:33:08] studying, they will probably pass. Then you have a threshold value at the 0.5, the middle usually. And your probability, if it's less than 0.5, the value is rounded off to 0, indicating the student will fail. So, if they're
[4:33:22] going to fail. This, of course, is ignoring the outliers of that one doesn't need any studying to memorize everything. That's not me, unfortunately. I have to study hard to learn new stuff.
[4:33:36] Problem statement: to classify whether a tumor is malignant or benign. And this is actually one of my favorite data sets to play with because it has so many features. And when you look at them, you really are hard to understand. You can't
[4:33:51] So, it gives you a chance to kind of dive into what data looks like when you domain of the data. But I also want you to remind you that in the domain of medicine, if I told you that my probability was really good at
[4:34:06] probability was really good at classifying things that say 90% or 95% and I'm classifying whether you're going to have a malignant or a benign tumor. it tested anyways. So, you got to remember the domain we're working with.
[4:34:20] know you're just going to go get a biopsy because, you know, it's that Just referencing the domain, it's important. It might help the doctor know where to look just by understanding what kind of tumor it is. So, it might help
[4:34:37] them or aid them in something they missed from before. So, let's go ahead back to the domain part of it in just a minute. So, use case and we're going to do our normal imports here. We're importing NumPy, Pandas, Seaborn, the
[4:34:50] Matplotlib library and we're going to do Matplotlib library in line since I'm let's go ahead and flip over there and get this started. So, I've opened up a new window in my Anaconda Jupyter notebook.
[4:35:04] don't have to use Anaconda for the Jupyter notebook. I just love the interface and all the tools in Anaconda brings. So, we got our import NumPy as NP for our NumPy number array. We have
[4:35:17] our Pandas PD. We're going to bring in Seaborn to help us with our graphs as SNS. So many really nice tools in both Seaborn and Matplotlib library. Then we'll do our Matplotlib library.pyplot as PLT and then of course we want to let
[4:35:31] ahead and just run that so it's all set up. data. I'm not creative today. Uh equals PD and not creative today. Uh equals PD and this happens to be in a CSV file.
[4:35:45] So, we'll use a pd.read_csv and I happen to name the file or renamed it data for p2.csv. You can of course write in the comments data set itself or go to the Simply Learn website and we'll be happy to
[4:36:00] supply that for you. And let's just open up the data before we go any further and let's just see what it looks like in a spreadsheet. So, when I pop it open in a local spreadsheet, this is just a CSV file,
[4:36:13] comma separated variables, we have an ID. So, I guess the categorizes for reference for what ID which test was done. The diagnosis, M for malignant, B options on there. And as we were going to try to predict is the M and B and
[4:36:29] tested. And then we have like the radius mean or average, the texture average, perimeter mean, area mean, smoothness. I don't know about you, but unless you're stuff, I mean, you can guess what concave means just by the term concave,
[4:36:47] means in the measurements they're taking. So, they have all kinds of stuff like how smooth it is, the symmetry. And these are all float quick and you'll see there's, I believe, 36 if I remember correctly in this one.
[4:37:04] take when they go in there and they take a look at the different growth, the tumorous growth. So, back in our data, and I put this in the same folder as the code. So, I saved this code in that folder. Obviously, if
[4:37:18] want to put the full path in there. And we'll just do um Pandas first five lines of data with the data.head. We run that, we can see that we have
[4:37:31] pretty much what we just looked at. We have an ID, we have a diagnosis. If we go all the way across, you'll see all the different columns coming across all the different columns coming across displayed nicely for our data.
[4:37:46] seaborn, which we referenced as SNS, makes it very easy to go in here and do a joint plot. You'll notice the very similar to cuz it is sitting on top of the plot library. So, the joint plot does a lot of work for us. And we're
[4:38:01] just going to look at the first two columns that we're interested in, the radius mean and the texture mean. We'll just look at those two columns. And data equals data. So, that tells it which two columns we're plotting and
[4:38:13] then we're going to use the data that we pulled in. Let's just run that. here. And there's all kinds of cool things on the texture mean and the radius mean. Obviously, the axes. You can also see
[4:38:32] show that for the radius mean. Where is the most common is radius mean come up and where are the most common texture is. That's what we're looking at the the on each growth it's average texture and on each radius it's average radius on
[4:38:47] there. Gets a little confusing cuz we're talking about the individual objects' average and then we can also look over here and see the the histogram showing us the median or how common each measurement is. And that's only two
[4:39:02] columns. So, let's dig a little deeper into seaborn. They also have a heat map. And if you're not familiar with heat maps, a heat map just means it's in color. That's all that means. Heat map I guess the original ones were plotting
[4:39:15] heat density on something and so ever since then it's just called a heat map. our corresponding numbers to put that into the heat map. And that's simply data.corr for that. That's a pandas expression.
[4:39:29] data frame. So, that's one of the cool tools in pandas for our data. And let's just pull that information into a heat map and see what that looks like. And you'll see that we're now looking at all the different features. We have our
[4:39:43] ID, we have our texture, we have our area, our compactness, concave points. And if you look down the middle of this chart diagonal going from the upper left to bottom right, it's all white. That's because when you
[4:39:57] compare texture to texture, they're identical. So, they're 100% or in this case, perfect one in their correspondence. And you'll see that when you look at, say, area or right below it, it has
[4:40:11] almost a black on there when you compare it to texture. So, these have almost no form a linear graph or something that you can look at and say that they're all scattered data. This is really just a really nice graph
[4:40:24] to get a quick look at your data. Doesn't so much change what you do, but an answer or something like that or you start looking at some of these individual pieces, you might go, "Hey, that doesn't match according to showing
[4:40:38] our heat map, this should not correlate with each other. And if it is, you're why? What's going on? What else is coming in there?" information on here. I mean, we can see from the ID,
[4:40:52] there's no real one feature that just says, if you go across the top line, that lights up. There's no one feature that says, "Hey, if the area's a certain malignant." It says there's some that sort of add up. And that's a big hint in
[4:41:08] the data that we're trying to ID this, whether it's malignant or benign, that's go, "Okay, we can't solve this with any one feature. It's going to be something that includes all the features or many of the
[4:41:22] solution for it. And while we're exploring the data, let's explore one exploring the data, let's explore one more area and let's look at data.isnull. We want to check for null values in our data. If you remember from earlier in
[4:41:37] differently where we added stuff up and summed them up. You can actually with pandas do it really quickly, data.isnull and sum it, and it's going to go across all the columns. So, when I run this,
[4:41:51] you're going to see all the columns come up with no null data. few steps, we've done a lot of exploration. We have
[4:42:03] looked at the first two columns and seen how they plot with the seaborn with a joint plot, which shows both the histogram and the data plotted on the XY coordinates. And obviously, you can do that more in
[4:42:17] with different columns and see how they plot together. And then we took and did the seaborn heat map, the sns.heatmap of the data. And you can see right here where it did a nice job showing us some
[4:42:30] bright spots where stuff correlates with each other and forms a very nice combination or points of scattering points. And you can also see areas that don't. And then finally, we went ahead and
[4:42:42] checked the data. Is the data null value? Do we have any missing data in there? Very important step cuz it'll crash later on if you forget to do this step, it will remind you when you get that nice error
[4:42:54] code that says null values, okay? So, not a big deal if you miss it, but it it's no fun having to go back when you're when you're in a huge process and 10 steps later and you got to go remember where you were pulling the data
[4:43:07] in. So, we need to go ahead and pull out our X and our Y. So, let me just put that down here. And we'll set the X equal to And there's a lot of different options here. Certainly, we could do X equals
[4:43:21] all the columns except for the first two cuz if you remember, the first two is the ID and the diagnosis. So, that certainly would be an option. But what we're going to do is we're actually going to focus on the worst the
[4:43:33] worst radius, the worst texture, parameter, area, smoothness, compactness, and so on. One of the reasons to start dividing your data up when you're looking at this information is sometimes the data will be the same
[4:43:49] data coming in. So, if I have two measurements coming in to my model, it might overweigh them. It might overpower the other measurements because it's information in twice. That's a little bit past the scope of
[4:44:01] this tutorial. What I want you to take away from this though is that we are our team in the back went ahead and said, "Hey, let's just look at the worst." So, I'm going to create a an array,
[4:44:13] and you'll see this array radius worst, texture worst, perimeter worst. We've I'm just going to put that in my X. So, this X is still a pandas data frame, but it's just those columns. And our Y, if you remember correctly, is going to be
[4:44:30] Oops, hold on 1 second. It's not X, it's data. There we go. So, X equals data, and then it's a list of the different columns, the worst of the worst. And if to have our answer for our Y, for the stuff we know.
[4:44:44] And if you remember correctly, we're just going to be looking at the diagnosis. That's all we care about is what is it diagnosed? Is it benign or malignant? And since it's a single column, we can
[4:44:56] just do diagnosis. Oh, I forgot to put the brackets over the There we go. Okay. So, it's just diagnosis on there, and we can also real quickly do like X.head if you want to see what that looks like, and Y.head,
[4:45:10] and run this. And you'll see um it only does the last one. I forgot about that if you don't do print. You can see that the the Y.head is just MMM cuz the first ones are all malignant. And if I run this, the X.head
[4:45:24] is just the first five values of radius worst, texture worst, parameter worst, area worst, and so on. I'll go ahead and take that out. I'll go ahead and take that out. So, moving down to the next step, we've
[4:45:37] built our two data sets, our answer and then the features we want to look at. In data science, it's very important to test your model.
[4:45:49] So, we do that by splitting the data. And from SK-Learn model selection, we're going to import train test split. So, we're going to split it into two groups. There are so many ways to do this. I noticed in one of the more modern ways
[4:46:03] actually split it into three groups and then you model each group and test it all kinds of and there's reasons for that, which is past the scope of this and for this particular example isn't necessary. For this, we're just going to
[4:46:17] split it into two groups, one to train our data and one to test our data and our data and one to test our data and the SK-Learn uh {dot} model selection, we have train test split. You could write your own quick code to do this
[4:46:29] up into two groups, but they do it for us nicely. actually do it in one statement with this where we're going to generate four this where we're going to generate four variables, capital X train, capital X
[4:46:43] test. So, we have our training data we're going to use to fit the model and then we have our Y train, so we're going to train the answer and then we have our test. So, this is the stuff we want to see how good it did on our model.
[4:46:56] And we'll go ahead and take our train test split that we just imported. And we're going to do X and our Y, our two different data that's going in for our split. And then the guys in the back came up
[4:47:09] and wanted us to go ahead and use a test size equals {point}3. That's test {underscore} size. Random state, it's always nice to kind of switch your random state around, but not that important. What this means is that the
[4:47:21] test size is we're going to take 30% of the data and we're going to put that into our test variables, our Y test and our X test. And we're going to do 70% into the X train and the Y train. So, we're going to use 70% of the data to
[4:47:35] train our model and 30% to test it. Let's go ahead and run that and load those up. So, now we have all our stuff split up and all our data ready to go. part. We're going to actually going to do our create our model.
[4:47:49] So, let's go ahead and bring that in from SKlearn. We're going to bring in our linear model and we're going to import logistic regression. That's the actual model we're using. And this is going to call that log model.
[4:48:02] Oops, there we go. Model and let's just set this equal to our logistic regression that we just imported. So, now we have a variable log model set to that class for us to use. And with most the models in the SKlearn,
[4:48:18] we just need to go ahead and fix it fit do a fit on there and we use our X train that we separated out with our Y train. And let's go ahead and run this. So, once we've run this, we'll have a model that fits this data, that's 70% of our
[4:48:33] that fits this data, that's 70% of our training data. that you can set on there. There's a lot of different choices you can make, but to let all the default set. We don't really need to mess with those on this
[4:48:46] particular example. And there's nothing in here that really stands out as super important until you start fine-tuning it. But for what we're doing, the basics will work just fine. And then let's We need to go ahead and test out our model.
[4:49:00] Is it working? So, let's create a variable Y predict. And this is going to variable Y predict. And this is going to be equal to our log model. And we want to do a predict. Again, very standard format for the SKlearn library
[4:49:13] is taking your model and doing a predict on it. And we're going to test Y predict against the Y test. So, we want to know what the model thinks it's going to be. That's what our Y predict is. And with that, we want the capital X X test. So,
[4:49:28] we have our train set and our test set and now we're going to do our Y predict. And let's go ahead and run that. And if we print Y predict Let me go ahead and run that.
[4:49:42] You'll see it comes up and it predicts an I prints a nice array of B and M for benign and malignant. For all the different test data we put So, that's pretty good. We're not sure exactly how good it does, but we can see
[4:49:57] that it actually works and is functional. It was very easy to create. You'll always discover with our data science that as you explore this, you prepping your data and making sure your data coming in is
[4:50:11] good. There's a saying, "Good data in, good answers out. Bad data in, bad answers out." That's only half the thing. That's only half of it. Selecting your models becomes the next part as far as how good
[4:50:26] fine tuning it depending on what model you're using. good this came out. So, we have our Y predict here log model dot predict X predict here log model dot predict X test.
[4:50:43] we're going to go from the SK learn dot metrics. We're going to import classification report. And that just reports how good our model is doing and then we're going to feed it the model data and let's just print this out.
[4:50:55] data and let's just print this out. And we'll take our classification report And we'll take our classification report and we're going to put into there
[4:51:07] what we actually know is true and our prediction, what our model side. And let's run that and see what that And let's run that and see what that does.
[4:51:19] have a precision for benign and malignant B and M. And we have a precision of 93 and 91, a total of 92. So, it's kind of the average between these two of 92. There's
[4:51:32] all kinds of different information on here. Your F1 score, your recall, your support coming through on this. flip back to our slides that they put together for describing it. And so, here
[4:51:46] we're going to look at the precision using the classification report. And you above. Some of them my numbers might be different cuz it does randomly pick out which data we're using. So, this model is able to predict the type of tumor
[4:52:00] with 91% accuracy. So, we look back here, that's you will see where we have benign and malignant, it actually is 92 coming up here. But, we're looking about a 92, 91% precision. And remember, I reminded you
[4:52:13] about domain. So, we're talking about the domain of a medical domain with very catastrophic outcome, you know, at 91 or 92% precision, you're still going to go in there and have somebody do a biopsy on it. Very different than if you're
[4:52:28] investing money and there's a 92% chance you're going to earn 10% and 8% chance you're going to lose 8%, you're probably going to bet the money cuz at that odds, money and in the long run, if you do that enough, you definitely will make
[4:52:42] money. And also with this domain, I've actually seen them use this to identify different forms of cancer. That's one of the things that they're starting to use these models for cuz then it helps the doctor know what to investigate. So,
[4:52:54] that wraps up this section. We're finally we're going to go in there and let's discuss the answer to the quiz asked in machine learning tutorial part Can you tell what's happening in the following cases? Grouping documents into
[4:53:07] different categories based on the topic and content of each document. This is an clustering can be used to group the documents by topics using bag of words approach. So, if you got in there that
[4:53:21] hopefully you had at least one or two examples like K means that are used for clustering different things, then give yourself a two thumbs up. B, identifying handwritten digits in images correctly. This is an example of classification.
[4:53:37] would be to extract digit dependent features like curvature of different digits, etc. And then use a classifier like SVM to distinguish between images. Again, if you got the fact that it's a classification example, give yourself a
[4:53:51] thumb up. And if you're able to go, "Hey, let's use SVM or another model for on it. C, behavior of a website indicating that the site is not working as designed.
[4:54:04] This is an example of anomaly detection. In this case, the algorithm learns what is normal and what is not normal, usually by observing the logs of the website. Give yourself a thumbs up if you got that one. And just for a bonus,
[4:54:18] anomaly detection? One of the ones I use it for in my own business is detecting anomalies in stock markets. Stock markets are very fickle, and they behave very erratical. So, finding those erratic areas and then finding ways to
[4:54:32] track down why they're erratic. Was something released in social media? Was something released You can see where knowing where that anomaly is can help you to figure out what the answer is to it in another area. D, predicting salary
[4:54:45] of an individual based on his or her years of experience. This is an example of regression. This problem can be mathematically defined as a function between independent years of experience and dependent variables salary of an
[4:54:58] individual. And if you guessed that this was a regression model, give yourself a thumbs up. And if you're able to remember that it used between independent and dependent variables and that terms, give yourself two thumbs up.
[4:55:11] Summary. So, to wrap it up, we went over what is K-means, and we went through also the chart of choosing your elbow method and assigning a random centroid to the clusters, computing the distance, and then going in there and figuring out
[4:55:27] through that loop until it gets the perfect centroid. And we looked into the elbow method to choose K based on running our clusters across a number of variables and finding the best location
[4:55:39] for that. We did a nice example of clustering cars with K-means, even columns to make it simple and easy to graph. You can easily extrapolate that and look at all the different columns and see how they all fit together. And
[4:55:52] we looked at what is logistic regression. We discussed the sigmoid function, what is logistic regression, and then we went into an example of classifying tumors with logistics. I hope you enjoyed part two of machine
[4:56:06] >> In this video, we are going to learn how to automate Google Sheets using n8n. It is a very powerful workflow automation tool. We will start by setting up a webhook trigger to receive data from external sources, then connect it to
[4:56:21] Google Sheets to automatically store and update data in real time. By the end of this tutorial, guys, you will be able to capture form submissions, automate data entry without manual work, or set up live workflow that runs seamlessly. So,
[4:56:34] whether you are a beginner or an automation pro, this tutorial is a must-see for you. First, understanding what exactly is n8n. So, I'm at the what exactly is n8n. So, I'm at the official website of n8n, and n8n is an
[4:56:47] open-source workflow automation tool that allows you to connect various applications like APIs and databases without writing extensive code. It provides a visual editor where you can create workflows by linking different
[4:57:01] nodes. Think of it an alternative to Zapier and Apache Airflow. But this tool has more control and flexibility and also it has self-hosting capabilities. Now if you see the key features of n8n are as follows. It has this amazing
[4:57:17] visual editor where you can drag and drop nodes to create workflows. No complex coding is required. It has over 350 plus integrations and it supports APIs, databases, messaging apps, AI tools and cloud services. For example,
[4:57:33] you can think of like Google Sheets, Slack, Telegram, OpenAI, AWS, GitHub, MySQL. So it has integration to these tools. And apart from that, it has self-hosting and cloud options. You can run it on your own server for data
[4:57:48] privacy, also available as cloud service. It has also conditional logic and data transformation. It supports if condition, loops, branching. You could also modify API responses, filter data
[4:58:01] and perform calculations. And guys, this feature is very, very amazing. For real-time automation, it has webhooks. So n8n can receive data from webhooks to trigger instant actions. For example, when a form is submitted in Typeform,
[4:58:17] I'll send an email via Gmail. So it can automate in real time. Now you will be wondering, how does n8n works? Now, let us try to understand how n8n works exactly. So you can think of n8n as a digital
[4:58:34] assistant that automates your task without requiring you to write complex codes. It basically works by connecting different application, services and database into sequence of steps called workflow. So if you go and login to the
[4:58:48] register for this n8n, you you're going to get a workflow template. Okay, you can go all over here. And you can
[4:59:03] and give a description about it and this could be your you know template code whether it is free or paid you can just click on this and after that I have reviewed my submission guidelines of N8N terms and services and then you can
[4:59:17] click this workflow. Now let us try to understand what exactly is a workflow. But before that let us try to understand nodes, okay? So nodes are tiny workers that do specific task. You can consider
[4:59:31] this something like that. And workflows are a sequence of nodes working together to complete a job. Like for example let us say we are sending an email when you get a new form submission. Imagine you have a Google form where people can
[4:59:44] contact you. So every time someone builds the form you want to automatically send them an email. So in N8N you can create a workflow like this. So a Google Forms node collects the new response and Gmail node sends an
[4:59:59] automatic email to the person who submits the form. So this is something like this. We'll be doing a one hands-on tutorial also to elaborate more about it. So this is one of the way how it works. Next there's a concept of
[5:00:14] triggers. So there are trigger-based execution mode also. How to start your workflow. So you have to also understand guys that N8N workflows don't run all the time. They only start when something happens. So there are three ways to
[5:00:28] trigger a workflow. So the first one is trigger-based execution, okay? Which is basically called as an event-driven. The workflow automatically starts when a specific event occurs. For example let's
[5:00:41] say a new row is added to the Google Sheets. Now the workflow starts to run. Or you can think about it something like that when a customer fills out a Typeform survey the workflow runs. Think of it like a doorbell. When someone
[5:00:55] rings a bell like for example someone submits a form or updates a database, the workflow wakes up and does its jobs. Next one, we have the schedule execution, or we call it as cron jobs.
[5:01:08] Now, here the workflow run on schedule, like daily, weekly, monthly. For example, every Monday at 9:00 a.m. a report is generated and sent via email. Or every hour, n8n checks for new tweets and saves them. Think of this like an
[5:01:22] alarm clock. You set it once and it automatically rings at the right time. Now, let us discuss about the workflow components, which are like the brains of the system. So, every n8n workflow is made up of
[5:01:35] different type of nodes, and each node has a special role, just like team members working on a project. So, whenever we trigger the node or start the workflow, so what happens exactly? First, there is a webhook node, which it
[5:01:49] starts someone wins, like for example, someone is sending a data to your API. starts at a specific time, like every morning at around 8:00 a.m. And then there is a Typeform trigger node, which it starts when someone fills out a
[5:02:04] Typeform survey. So, we'll try to cover all these things in this today's tutorial. Now, there are some also nodes, like we call them as action nodes, which basically perform task. So, these nodes do not actually work in your
[5:02:17] workflow. For example, let's say send an email node or send an email via Gmail or Outlook. Or there is an HTTP request node, which fetches a data from an API, like a weather update or something. Then there is a Google Sheet node, which like
[5:02:31] adds a new data to a spreadsheet. So, these are called as action nodes, which basically performs task. Then you could also, you know, understand a bit about, let's say, logic nodes, which adds intelligence. So, these nodes allow you
[5:02:46] to add conditions and decision in your workflow. For example, let's say if node checks a condition is met. Like for example, is the email from a VIP You can imagine something like this. Then there is a switch node which routes
[5:03:01] the workflow based on different cases. Like for example, the payment example is the best one. Like if the payment is failed, send a reminder. Otherwise, if it's successful, send a confirmation. And then you could also
[5:03:14] find out the loop node which repeats an action multiple times like sending a follow-up email every 3 days. So guys, these were some of the logics nodes which basically adds intelligence. And finally, we have the data nodes
[5:03:30] which store and retrieves the information. These nodes basically help like Google Sheet node which helps you spreadsheet or MySQL node which stores customer data in a database or Airtable
[5:03:44] node which basically updates information in a table. Think of it like filing cabinet. It organizes and stores important documents or data for future use. So putting up all together, let's say you run an online store and you want
[5:03:58] to notify your team on Slack when a new order is placed in Shopify. So here is how your n8n workflow would look like. There is a trigger node like Shopify will trigger like it is going to detect a new order. Then the action node is
[5:04:12] going to come up which is going to fetch order details from Shopify. Then there is a logic node. If the order is above 1,000 US dollars, it marks it as a high priority. An action node which basically sends a Slack message to your sales
[5:04:27] team. So this is like the entire process which runs automatically saving your time and effort. Now guys, you would be wondering why should you use n8n? See guys, there is no need for coding skills. You can just drag and drop
[5:04:41] nodes. It saves a lot of time by automating repetitive task. And it works with hundred of apps and APIs like Google, Slack, Shopify, ChatGPT, etc. It also has self-hosting option which ensures privacy and full control. And
[5:04:56] finally, it is free and open source unlike some other tools which have some strict limits. Now, we are going to move forward and do a small hands-on to make you more understand about this platform. So guys, now let us create an automation
[5:05:11] using n8n. So, first click on the first step and let's say we want to use a trigger, let's say on a webhook call. So basically, when we will be sending any HTTP request like get, post, put, or delete, it is going to trigger our
[5:05:26] webhook on n8n. So, just click on this and you could see our method is, let's and you could see our method is, let's say, post and we want to post a data on say, post and we want to post a data on using this post request. And after
[5:05:40] selecting this, guys, just click for listen for test event. Let's test our webhook API. Copy this and then you can go to this online tool which is called as reqbin, okay? And it is just very simple tool
[5:05:55] where you could just do one thing. You could paste the same URL which you have copied it from here. Okay? Now, after going all from here, what are you going to do? Just click on the send request. Now, you could see we have got a
[5:06:09] successful message that the workflow has started. But how do we sure of that this thing has working because in the output section, you will be seeing these kind of output. Okay? So, our webhook API is
[5:06:23] working very much fine. Now, what next thing you could do? So, the next thing, let's say, let us add a node all over here. So, for example, let's say, let's add a Google Sheet, okay?
[5:06:43] say append a row in a sheet or we want to update a row. So, now you could see all over here we have this option called append or update a this option called append or update a row from this list. And uh, let's say we
[5:06:56] have this document called monthly expense. And you could also choose it, expense. And you could also choose it, let's say from uh, expense and we want to add uh, these properties. What values to send? Suppose name, we could set the
[5:07:09] Neil and all over here, that's one thing. Okay? Then uh, what we could do, we could uh, add another option, like what would be
[5:07:21] the cell format? And we could add the next property, let's say as email. next property, let's say as email. Email, let's say
[5:07:33] Okay? Now, we could test these step. Okay? Now, you could also say let's add a column, okay, to name, okay? And uh, let's say we could add just one value, then we could add another column,
[5:07:49] value, then we could add another column, let's say email, okay? Let's uh, another column. Okay? So, we had only two columns that I have actually created
[5:08:02] all over here, name and email. So, that's why our workflow is showing the same. And also guys, there is one point to note, you have to also connect to your given applications, okay? So, I'll just show you how you could do it. But
[5:08:16] okay? So, column manually. And let's test the given step.
[5:08:30] given step. Let's wait for the output. And if I open uh, Google Sheet, you could see we can see you know uh, the name email and it's already mentioned here. So this is like how you are basically creating your
[5:08:46] workflow. So if you go back to your canvas you could see our workflow has been created using the web a web hook call. Similarly you can trigger a lot of ways. Suppose you could add our next node like you could add AI and and all
[5:09:02] those things and you can create this automation using any tool. Guys this is a very amazing tool and this automation tool is definitely going to save a lot >> So we will be discussing about what
[5:09:15] exactly is LangChain, LangGraph, LangFlow and LangSmith and where can we use them and what makes them different from others. So this is the major agenda for today's session. Let's begin with LangChain.
[5:09:30] So what exactly is LangChain? LangChain is an open source framework designed to help developers build applications powered by large language models by providing tools and abstractions to simplify the process of
[5:09:44] integrating LLMs with external data sources and workflows. Now imagine you're building an AI application where an LLM model like GPT-4 is needed to query. Llama 3 or other versions to refine that
[5:10:00] response. A custom-built AI agent to decide whether it can answer the query without a hiccup or if it has to fetch external sources to generate a response. Most importantly a solid memory to store all previous user interactions. Without
[5:10:17] LangChain managing all these components manually would require extensive coding. You need to write a custom code snippet to handle API calls to your LLM like GPT, Llama, create and maintain a dedicated memory space, and most
[5:10:32] importantly, a fully customized decision-making capable AI agent. Yet, opportunity to scale it up, the possibilities are questionable. This
[5:10:45] approach leads to a lot of boilerplate code and makes the system complex. LangChain simplifies this by providing an open-source framework that helps developers build applications powered by large language models. It enables
[5:10:59] training prompts, interacting with external data, and adding memory for contextual understanding. Now, let's discuss some of the key features of LangChain. Firstly, LLM support. It works with both closed-source GPT-4 and
[5:11:12] open-source models like Llama 3 with simple API integrations. Prompt templates. It allows dynamic prompt customization instead of hardcoding Third one is chains. It helps connect multiple tasks like querying an LLM,
[5:11:27] retrieve data, and processing those responses in a seamless workflow. Fourth one is indexes. Supports document loaders and vector databases to pull in external data. First one is memory. It enables applications to retain past
[5:11:42] interactions, improving contextual awareness. And lastly, the agents. It uses LLMs as reasoning engines to dynamically decide the next step in the workflow. With LangChain, developers can efficiently build AI-powered
[5:11:56] applications without reinventing the wheel. You can have a detailed briefing on LangChain on the official website. Also, should you wish to try it out, take a quick look at quick start page to steps to follow to set up the
[5:12:10] environment and run LangChain in your local system. Now, let's proceed with the next one in this tutorial, which is LangGraph. So, what exactly is LangGraph? By definition, LangGraph is a library
[5:12:26] within the LangChain's ecosystem. It is a framework for building and managing complex stateful multi-agent LLM applications by modeling workflows as graphs, where nodes represent tasks or states, and edges represent a transition
[5:12:39] between them. Now, let us imagine this. You are in a situation where you need to work with not one, but multiple agents in parallel. But, LangChain can just assist you with one. You need much more
[5:12:54] enhanced platform for this. That's where LangGraph comes into the picture. LangGraph is built upon the existing LangChain and designed for managing agents and multi-agent workflows. While LangChain is excellent for chaining
[5:13:08] prompts, LangGraph excels at handling structured interactions between multiple agents. If you are building a system where multiple agents collaborate, such as a research assistant or an automation workflow, LangGraph is a great choice.
[5:13:23] So, let's discuss these core components of LangGraph. First one, state. A state is a shared data structure that stores user inputs, agent outcomes, and workflow's history. Now, the next one is nodes. You can have one node or also
[5:13:40] multiple nodes. These nodes are individual components or actions that execute LLM calls, run functions, or interact with external tools. Now, let's Edges are nothing but connections between nodes that define execution
[5:13:57] flow. Unlike traditional directed graphs, nodes in LangGraph can dynamically decide which node to call next. That means the control flow can go in both ways. LangGraph is ideal when you need agents to interact in cyclical
[5:14:11] workflows or collaborate on complex tasks. You can have a detailed briefing on LangGraph on the official website. Also, should If wish to try it out, take a quick look at the quick start. Using LangGraph is straightforward. You just
[5:14:25] need to install the package, import it onto your project, and then using the provided classes, you can straightly access LangGraph. Now, let's proceed to the next one, which is LangFlow.
[5:14:38] Now, what is LangFlow? LangFlow is a visual, low-code framework built upon top of LangChain, enabling developers to easily design and test AI applications, especially those involving agent systems and retrieval augmented generation, or
[5:14:54] RAG. Now, imagine building AI-powered applications like chatbots or data processing tools without writing any code. LangFlow makes this possible with a drag-and-drop interface. LangFlow is built upon top of LangChain and provides
[5:15:09] a visual interface for designing LangChain workflows. It is perfect for prototyping LLM applications, allowing users to quickly design workflows, chains, and agents, and test them. However, it is not intended for
[5:15:22] production use, but it is excellent for MVPs, which is minimum viable products. LangFlow is designed to simplify AI-powered app development by providing a drag-and-drop interface. It enables developers to visually design,
[5:15:36] experiment with, and test LangChain-based workflows. Now, what makes LangFlow different from LangChain and LangGraph? Why use LangFlow? So, there are three major reasons. Firstly, no coding required.
[5:15:51] Perfect for prototyping AI applications without having to write any code. Second one, visual workflow builder. You can easily design chains, agents, and integrations. Thirdly, quick prototyping. Best suited for MVPs,
[5:16:06] experimentation, and non-production use cases. So, there are two main ways to use LangFlow. First one, via DataStax, a paid cloud-hosted option. Second one, self-hosted. Install locally or on your own cloud server.
[5:16:22] So, once deployed, LangFlow provides a UI where you can drag and connect various components enabling you to build and test AI workflows seamlessly. You can have a detailed briefing on LangFlow on the official website. Also,
[5:16:37] should you wish to try it out, take a quick look at quick start to work on environment. You just need to install the package, import it onto your classes. Now, let's move on to the last one,
[5:16:51] which is LangSmith. So, what exactly is LangSmith? So, LangSmith is a platform developed by LangChain designed to streamline the development, testing, debugging, and monitoring of LLM applications, offering
[5:17:06] tools for observability, evaluation, and prompt engineering. The LLM GenAI AI agents, all the hype was a huge hit. Everybody is interested in developing a custom AI agent to automate and optimize all the redundant
[5:17:20] CRM tasks, and much more. But, building an LLM-based application or AI agent is just the beginning. Monitoring, testing, and optimizing are equally crucial.
[5:17:32] LangSmith is designed to help developers at every stage of an AI application's life cycle, from prototyping to production. Now, why use LangSmith? So, there are three important reasons. Firstly, performance monitoring. It
[5:17:45] tracks API calls, token usage, errors, and latency. Second one, debugging and evaluation. It ensures models return expected results and avoid unnecessary API costs. Lastly, independent framework. It works with any LLM
[5:18:00] framework, not just the LangChain-based ones. If your application requires extensive monitoring, debugging, or optimization, LangSmith is a great tool. However, for simple applications without complex tracking means it might not be
[5:18:13] necessary. To integrate LangChain, install the package, set up the environment variables, and use traceable annotations in your code. The LangSmith dashboard provides insights into performance metrics, including token
[5:18:26] usage, call trends, and error rates. For more details on LangSmith, visit the official page of LangSmith. Also, should you wish to try it out, take a look at the quick start to work on setting up LangSmith in your local
[5:18:40] the package, import it into your project, and start using the provided >> Something big is happening in the world of AI, and it's shaking up the industry. No, it's not another ChatGPT update or a new release from OpenAI. This time, it's
[5:18:56] a brand new player on the scene, DeepSeekR1, an AI model that's already outperforming the best-known models in ways that have experts seriously being talking. But why is this causing such a stir? Well, it's because DeepSeekR1
[5:19:12] doesn't just answer questions like other AIs. It thinks. And I mean really thinks. It breaks down complex problems step-by-step, showing you exactly how it arrives at the solution. You just don't get an answer, you get the entire
[5:19:27] process from start to finish, that too in real time. This is the kind of AI that doesn't just solve problems, it solves the way we do. Now, here's the twist. DeepSeekR1 isn't from one of the usual AI giants like OpenAI or
[5:19:42] Anthropic. It was developed by High Fly Capital Management, a hedge fund that's quietly been making waves in the AI world. In fact, their new DeepSeekR1 light preview is already making headlines for its stunning performance
[5:19:56] on tasks like mathematical reasoning, logical inference, and real-time decision-making, areas where even the best of the OpenAI's models often fall short. But why is this model such a game-changer? It's because in 2024, the
[5:20:11] world is demanding AIs that can think logically and reason through problems. DeepSeek's ability to show its work is what sets it apart. While other AIs give you answers, DeepSeek lets you watch its thought
[5:20:25] process unfold. Think of it as the AI that doesn't just solve problems, but teaches you how to think like it does. And if you think that's all just a hype, let me tell you, Forbes isn't the only one buzzing about it. Even experts
[5:20:39] across the industry are pointing out that DeepSeek is outperforming models from heavyweights like OpenAI and Anthropic on test that measures complex reasoning. It's already winning in some major AI benchmarks, and it's only going
[5:20:53] to get better. So, why is DeepSeek-R1 such a big deal right now? And what does it mean for the future of AI in 2024 and beyond? Well, in this video, we will explore DeepSeek-R1 starting with its unique
[5:21:06] capabilities and how it stands out from the other AI models. We'll also test its math reasoning and problem-solving skills, comparing it with models like OpenAI. Then, we will examine the real-world application in fields like
[5:21:19] So, let's dive in and see how it's shaping the future of AI. So, at first, let us understand what is exactly DeepSeek-R1 like to do. As you can see this tweet post, so DeepSeek-R1 like to do is an AI tool which is similar to
[5:21:35] ChatGPT, and it's created by a Chinese company which is known as Deep Seek. The company announced this new model on X on November 20th and shared a few details. So, I'll show you the documentation page of here. So, this is a documentation
[5:21:51] page of Deep Seek API, and DeepSeek-R1 like to do is meant to be really good at solving problem reasoning in maths, coding, and logic. It shows you how to think step by step so that you can understand how it comes up with answers
[5:22:05] and which help people people trust more. So, you can try out me on the website you. So, you just have to simply go to Google
[5:22:17] and type over here chat.deepseek.com. redirected to DeepSeek's page. So, simply me, I'll just log in with the
[5:22:31] Google account. And here you can see this is the interface of our DeepSeek our light model. And here we will enable this DeepThink preview, which is a new feature over here. And now let's start with our test and start
[5:22:47] testing whether DeepSeek this model is really that effective or worth the hype or not. So, first we'll be doing the strawberry test, okay? So, let's see what exactly is the strawberry test. So, let's start
[5:23:00] with a very simple question, which is how many times does the letter R occur in the word strawberry? So, this is our question over here and let us just check So, I didn't expect such a long
[5:23:14] reasoning process to what seems like a straightforward task. I thought that after counting the letter R and identifying the position of the word, it would have stopped there. But what's interesting to me is that it didn't
[5:23:26] stop. It double-checked the counting a couple of times and even considered things like how people might pronounce or spell the word differently, which I think is a bit of pedantic, especially the pronunciation part. But then that's
[5:23:40] how it does show how careful and thoughtful it is. So, it just explained us every step-by-step and what process we can go through it. So, let's move on to our second question, which would be a mathematical reasoning question. So, our
[5:23:56] next question would be if a triangle So, if a triangle has sides of 3, 4, 5, what would be its area? So, let's just check. So, as you can see our Deep Seek our
[5:24:10] model is getting our answers. And it performs the checks I predicted, although in a different order. But then, it's using all the steps and all the formulas and what would be the best to go with this question. So, both
[5:24:25] the explanation and the output are particularly clear and easy to follow, which makes me think that this would be a fantastic model to embed in math students assistant. For example, this is a particular use case. Maybe the thought
[5:24:38] process could be shown first and the student could interact with acknowledging whether they understood it or not. So, that's a pretty fast answer. Let's move on to our third
[5:24:51] question. So, now let's ask our Deep Seek model a geometrical type of question, which is a bit tougher and let's see what answer would it generate for us. And how it is it taking the logic
[5:25:04] building and the process. So, as you can see it's taking all the formulas and also it is considering the other uh probabilities as well. So, we'll just wait a few seconds.
[5:25:17] So, as you can see it has written later messed up in a denominator. So, it is even checking what mistakes has it generated, which is really good. It is just like as if somebody is solving this mathematical problem.
[5:25:30] But then in the form of AI as well. That's really interesting. So, now as you can see it has given us the final answer and also step-by-step approach, which is well-known geometric formulas and applied them directly,
[5:25:45] which makes the reasoning very easy to follow. So, the output is very clear and examples, it's been impressive to see how it consistently double-check
[5:25:57] calculations using different methods. The thought process is always deeply logical and very easy to understand. So, for somebody who wants to actually learn problems, this model will give a very detailed
[5:26:12] explanation. So, I would highly suggest using this. So, let's now move on to our next question, which which would be a coding test. So, the first question would be uh implement a function in Python that finds the longest
[5:26:26] palindromic substring in a given particular string. The function should particular string. The function should have a time complexity better than O n ^ 3. Big O of n ^ 3. So, let us just see what and what is the time complexity
[5:26:41] will it generate the answer for us? The code, let's just check it out. So, as you can see, it has given us the solution code here, and also the explanation part. And whatever the function it is using,
[5:26:54] it has briefly mentioned the explanation about it. And I think that the model did a very great job solving the problem and finding the longest palindromic substring. The approach was smart,
[5:27:08] efficient, and clearly explained. So, instead of just brute force method every possible substring, which would be slow, it used a very clever expand around center technique. And this method handled both odd-length palindromes and
[5:27:23] even-length palindromes as well, which was really clear and effective. So, that was for the coding part. So, now let's move on to a logical reasoning type of question, and let's see what answer does it generate. And uh
[5:27:39] and we will also compare it with the OpenAI model here as well. So, let's quickly check it out. Our question here is a man has to cross a river with a wolf, a goat, and a cabbage. His boat can only carry himself and one other
[5:27:53] thing. If left alone, the wolf the wolf would eat the goat and the goat would eat the cabbage. How can he get everything across safely? So, let's quickly check out the answer. So, as you can see, it's thinking of the end of
[5:28:07] reading the answer for us. Also, for the Deep Think messages, you Also, for the Deep Think messages, you can use 50 messages free per day. So, we have already used five texts and we have 45 messages
[5:28:19] left for today. So, as you can see, it has given a clear, detailed explanation. has given a clear, detailed explanation. And it solved the problem really very well. I think it carefully went through the rules and checked different
[5:28:34] possibilities. It understands that some pairs like the wolf and the goat or the goat the cabbage can't be left alone. It also reviewed the constraints at the starting. It also reviews the constraints at the start.
[5:28:47] Starting from this, it looks at what would happen if the man takes each item across the river first and works out if it creates any problem. What I find really nice is about how the model adjusts its plan when something doesn't
[5:29:02] work. For example, when it tries taking the wolf first, it realizes talking to the wolf first but realizes at the same time that it causes trouble and then rethinks the step. This trial and error method looks very similar to how we as
[5:29:17] humans might solve the puzzle ourselves. So, in the end, the model comes up with a very right solution and explains it in a very clear and step-by-step way. So, as you can see, I've given ChatGPT this question and this is the answer it has
[5:29:32] generated. But then, it wasn't that detailed as compared to Deep Think detailed as compared to Deep Think model. So, here I would rate Deep Think model more because it gave me a very clear explanation with all the details
[5:29:47] that you can clearly see the difference out here. So, for anyone who wants to understand from depth, then this Deep Seek model would be a great help. So, let's quickly have a look on the Deep Seek our own light review benchmarks.
[5:29:59] So, we have tested the model in real world conditions, but how does it stack up against other AIs on benchmarks like AI and math and competitive programming languages. So, you have this on the screen. We have compared it with AI and
[5:30:15] math, GPT, and diamond, code forces, light code bench, and also zebra logic. So, according to the benchmarks, Deep Seek our own light review blows the competition out of the water when it comes to math and logic problems with a
[5:30:30] pass rate of 52.5% on the AI and 2024 challenge. It is really ahead of Open AI's own preview and surpasses models like GPT-4 on
[5:30:43] complex math tasks. So, with this we have come to the last part of the section and here we will be comparing that who really wins. At first, you have seen that we have compared with the Open AI model. So, let's just quickly test
[5:30:56] some questions and just check it out that with Open AI which one performs much more better. Now, we will compare Deep Seek versus Open AI own model with some command prompts to see who really wins in different categories. So, here I
[5:31:12] have this Deep Seek model. So, my first command for this test is a math problem and it says solve for X the quadratic equation 3x^2 + 5x - 2.
[5:31:24] So, let's quickly see this what answer is it giving and we will be comparing it with the Open AI own model as well. So, as you can see it is giving us all the
[5:31:36] available solutions using different methods as well. And see the first formula. It's basically giving you the full overview of all the methods in which you find easier and you can use that. And
[5:31:51] here is our final answer. So, let's compare it with the OpenAI Oven model as well. So, here I have this question and I'll just check the answer. It's taking a bit of time, but let's wait for me. So, as you can see, it has
[5:32:07] only used one formula, which is the quadratic formula. And here we have the answer for this thing. But what I found is it provided a correct solution, but then it lacked uh whereas our Deep Seek model, it provided
[5:32:24] So, I think our Deep Seek wins for this math question as well. For the second comparison, I have a coding problem. So, let's check for both. Uh first, we will be checking for Deep Seek
[5:32:38] And the coding problem is write a Python function to check if a given string is palindrome or not. And I'm waiting for the answer here. So, as you can see, Deep Seek Carbon 9 Review model has provided us a correct
[5:32:52] solution along with a detailed breakdown of why each part of the code works as it of why each part of the code works as it is does. So, let's check for the Oven model as well. So, now as you can see for the coding task, uh the Oven model
[5:33:06] has given us the solution, but then it provides a working solution with clear code, but it doesn't explain the logic behind it. Whereas the Deep Seek model has provided the logic and also the solution with each detailed breakdown.
[5:33:20] would be Deep Seek Carbon 9 Review because it provides better and clearer explanation. Now, let's move on to the third test, which would be the logical third test, which would be the logical reasoning. So, here is my question for
[5:33:33] logical reasoning, which is if a man has five apples and gives three away, how many apples does he left with? So, let's check for the Deep Seek model. So, now you can see it has given us the answer and also the correct answer, but it has
[5:33:47] explained each and every logic to go through this question and why have we used this particular logic as well. So, that's a pretty uh breakdown detail step-by-step explanation. Let's check for the OpenAI Open model as
[5:34:03] well. So, here I have my question and let's wait for the answer. See, it has given us a straightforward answer, which is approx to apples, but then it hasn't mentioned the logic behind it, same as it was saying for the coding task. I
[5:34:18] think uh since both of these models have provided the accurate correct answer, but here also the winner would be Deep Seek because it has provided with a lot of explanation and transparency as well. So,
[5:34:31] this is the wrap-up of the video today and if you're looking for an AI that excels in reasoning, problem-solving and transparency, it's definitely a contender. While OpenAI and other models are fantastic, Deep Seek brings
[5:34:44] something new to the table. Step-by-step logic, clear explanation with an edge challenges. >> Hey everyone, welcome to Simply Learn's YouTube channel. In this session, we will learn how to run Deep Seek locally
[5:34:58] in Windows command prompt. But before that, let's take a look at the Deep Seek and all the hype about it. Deep Seek, the latest in generative AI, is making waves with its advanced reasoning and problem-solving capabilities. With
[5:35:12] problem-solving capabilities. With global AI adoption accelerating, Deep Seek emerges as a powerful contender, seemingly outpacing its rivals in efficiency and depth. Built on cutting-edge architecture, it excels in
[5:35:25] complex reasoning, coding, and multilingual understanding. AI is the new genius, and Deep Seek is pushing its boundaries further with unmatched performance. Its ability to handle intricate prompts with precision makes
[5:35:39] it a game-changer. Unlike traditional models, Deep Seek leverages superior contextual awareness for human-like responses. Its efficiency in real-world applications refines productivity and automation. Businesses and developers
[5:35:54] are rapidly integrating it for high-level decision-making. That's it. If these are the type of videos you'd like to watch, then hit that like and get notified. Now, before we get started, we need to check the space
[5:36:07] left. Let me explore the file explorer once. So, you can see the downloads is almost full. Now, we have another volume D, which has a little bit of space. Let's use that. Now, let's change the location of downloads to drive D. For
[5:36:22] that, go to downloads, open properties, and here go to location. Maybe if you want to create a new file, that's also okay. Let's try new file, that's also okay. Let's try that. So, here, let's create a file,
[5:36:36] and I'll name this as temporary downloads. avoid confusion. And this might ask for some permissions. an application, something like that, and you set environment for that, right? So,
[5:36:52] let's see that's the one. So, automatically, the computer will change Just in case if you have any other applications running from downloads directly. Meanwhile, while these particular files are copying, let's wait
[5:37:07] for a moment. Let's minimize everything and open a new tab on Google. So, to run Deep Seek locally, we might want to use a few things. Firstly, Ollama. So, let's download Ollama.
[5:37:20] Go to the Ollama thing in the third link. That's the one we need. Uh we need to download Ollama into our local system. For that, let's explore the models. So, this is the first one,
[5:37:32] looking for. So, before downloading that, let's enable Ollama. For that, we need to download and install Ollama. Download for Windows. So, the downloading and installation might take
[5:37:44] it. And we have started the download. Now, we will explore the DeepSeek before uh you know, all this download thing settles down. Let's explore a little bit about the DeepSeek models. So, here you
[5:37:58] about the DeepSeek models. So, here you can see we have 7B, 1.5B, 8B, 14B. And let's also view all. So, the B stands for billion parameters. So, DeepSeek has for billion parameters. So, DeepSeek has a lot of models based on the learning
[5:38:13] parameters, right? A couple of them have 8 billion. 671 is the highest, I guess, 8 billion. 671 is the highest, I guess, 400 GB. And uh because of space constraints, we will go with the first one, which is 1.5 billion parameters
[5:38:26] one. You can choose the model based on your space in your computer. So, I'll go with the 1.5B one. So, we'll just click there and you'll get all the drop-down options. Go with the one you want. And so, that, just copy that code there.
[5:38:41] Ollama run DeepSeek-R1 {double colon} one .5B, right? That's the one. And let's open the command prompt. Okay, before that, we need to install Ollama first. So, the file has been
[5:38:55] downloaded successfully. Navigate to the downloads location. Start the setup. It setup is completely done, then we will open the command prompt. And then, copy-paste that particular code. Then, everything will be automatically
[5:39:10] downloaded and loaded for you. So, DeepSeek will be ready to run locally on So, what's the benefit of running DeepSeek locally? Basically, when you want to communicate with your generative AI, you might want the internet
[5:39:24] connection with you. If you are able to download and set the generative AI, in our case, DeepSeek, in your computer, then it is beneficial for you so that it then it is beneficial for you so that it can run without the internet connection.
[5:39:38] It might take a little time. Don't worry. Once the setup is done, look at the bottom right corner. You'll get a slight notification, a smaller notification which says Ollama is set up and running,
[5:39:51] similar to that. You can just get Ollama is running now, something like that. Just a couple of moments while the final settings are done.
[5:40:05] Close it. Now, check the tray there. Should be having an Ollama logo present right there. There you go. That's the is running. Now, to check, you can also check the
[5:40:20] logs of Ollama which are presently in your computer. Just for cross-verification. Also, make sure that to copy that Ollama run DeepSeek R1 code there to run it on your command prompt. Just press Windows icon
[5:40:34] with R to have the run command, and you can type CMD to execute the command There you go. Now, make sure to copy that code. Once the Windows command prompt is active, you just have to paste it and
[5:40:48] hit enter. Command prompt is loading. It might take a little glitch. Let's wait until it shows up. Let's try again. CMD, hit okay. Let's
[5:41:01] prompt pops in. activities. There you go. We have multiple prompts. Let's close all the other ones which we don't need. I think it's still loading this close
[5:41:14] this as well. Let's give it a retry. CMD okay and let's meet. So here is the we were looking at earlier. Let's close the log file which we don't need it. Yeah, the command prompt is active paste enter.
[5:41:28] Now you should be getting something called manifesting. So that means the manifest. There you go. So based on the model that we chose which is 1.5 billion parameters it's
[5:41:43] typically a smaller version of the DeepSeek that we were looking at. The biggest one was 675 billion parameters which is close to 400 GB. If you have enough space in your PC then you can go with the highest version. So due to
[5:41:57] smaller one. The procedure is same. You just have to The procedure is same. You just have to paste the code and run. So at the last where you have 1.5 B right in place of that you'll be having 675 B. That's the
[5:42:10] only difference. It might take a little bit of time to download you're going with the higher package a bigger package then the download time bigger package then the download time might even end up more than 24 hours.
[5:42:24] And if you don't have enough space it might collapse in between. So I I'd select the one which suits with your memory space. Now the files are downloading. It might take a little time. Let's wait for that.
[5:42:38] third. So I think it's downloading in terms of packages. So we have the fifth package. Verifying. There you go. Successfully downloaded. Now
[5:42:51] we can type in a message. Let's say let's type hi DeepSeek or hello DeepSeek. Verify. Verify. So hi DeepSeek. So you can see when you
[5:43:04] thinking mode right. You can see it is also thinking. Now, let me type my request here. So, it also said as "Hello, how can I assist you?" So, my request is I wish to learn neural networks. So, you can see that when you
[5:43:21] open Deep Seek on Windows browser or Google browser, Firefox, any of the browsers, it'll show you the thinking process, right? Here also you can see that thinking process, start of thinking and end of think. The only thing is
[5:43:34] unable to display the thought process over here. But, fine. Now, let's continue with the command prompt, which is I wish to learn neural networks and is I wish to learn neural networks and hit enter. Now, let's say it will help
[5:43:46] you to learn neural networks in a stage-by-stage process in the learning order. What exactly is neural networks, what are the key components of neural networks, neurons, activation, weights, and biases, and everything, right?
[5:43:59] Similar to ChatGPT or even other competitors in terms of generative AI. And it is also giving you procedures and sample codes here, right? If you wanted to execute these codes on Jupyter notebook, then you can just import NumPy
[5:44:13] as NP, from TensorFlow, etc. And also you have some sample codes for generating a model, how to execute that, everything. So, I guess that's the end of the line for the prompt.
[5:44:27] Now, this was the trial run on how to run Deep Seek locally on your Windows command prompt. Now, you can also develop a completely new user interface and experience for your Deep Seek to run on locally. So far, so good. For the
[5:44:41] sample purpose, this is good. Now, let's type in "Bye" and see how it responds to the text. There you go. It might be confused, but it might give you an alternate approach. So, it looks like your message got cut
[5:44:56] off. Okay, could you clarify? Today we are exploring Hugging Face, an amazing tool that makes working with language and AI super easy. If you're curious about using advanced technology to understand or create text, this video is
[5:45:10] perfect for you. Hugging Face is a company that helps people use AI models for language tasks like translation, text analysis, or even generating new text. They have created a library called Transformer, which comes with
[5:45:23] pre-trained models, so you don't have to build everything from scratch. It's simple, powerful, and perfect for developers, researchers, or even beginners. So, in this video, I will show you three cool things you can do
[5:45:36] with Hugging Face. Speech to text, turn spoken words into written text easily, great for captions or voice apps. Second one is sentiment analysis. Find out if a text is positive, negative, or neutral. Helpful for understanding reviews or
[5:45:50] comments. The third one is text generation. Create human-like text that sounds natural. Perfect for chatbots or creative writing. I will also explain some important basics, like how pipelines make using Hugging Face models
[5:46:04] super easy, and how tokenization helps AI understand text. By end of this video, you will know how to use Hugging Face to start building your own projects. It's simple, fun, and really powerful. So, without any further ado,
[5:46:17] let's get started. So, welcome to this demo part of this video. three things. First thing, speech to text recognition. The second thing, text generation from a particular sentence or
[5:46:33] a word. And the third thing we will do sentiment analysis. Okay? So, we will perform this one by one. So, let's start with something cool, speech to text recognition. Okay? Using Hugging Face.
[5:46:47] recognition. Okay? Using Hugging Face. So, first I will write rename this. So, first I will write rename this. I'll write here Hugging Face I'll write here Hugging Face speech to
[5:47:00] Okay. Yeah. So, first I will install Yeah. So, first I will install transformers library. It is already installed on my system, but again,
[5:47:12] just for you guys, I'm doing this. Okay. PIP install transformer. So, now let's see what is transformer. So, transformers is a powerful Python library created by Hugging Face. So, that allows you to download,
[5:47:26] manipulate, and run thousands of pre-trained open-source AI model. So, as you can see, requirement already satisfied. It means like already installed. So, these transformer models cover multiple tasks across, you know,
[5:47:42] modalities like NLP, natural language processing, computer vision, audio, and multimodal learning. Like many things. Fine. Fine. So, now I will do from transformers
[5:47:58] pipeline. Okay. So, now let's run it. So, now what is pipeline? Okay. So, a transformer pipeline describes the
[5:48:10] flow of data from origin system to destination system. Fine. Because you already know transformer means you can run or manipulate thousands of pre-trained open-source AI
[5:48:22] So, pipeline describes the flow of data from origin system to destination system and defines how to transform the data along the way. Okay. So, let's check the
[5:48:37] versions. Transform Transform versions.
[5:48:55] That long time. Okay, why it is coming? Import transforms. formers formers Yeah. So, we have currently 4.42.4
[5:49:09] version of the transformer. Okay, so I am using Google Colab. Okay, so you can use either Jupyter Notebook, Visual Studio Studio or Colab.
[5:49:21] Okay. So now, let's import libraries. So now, let's import libraries. Import librosa. So, what is librosa? So, librosa is a Python package for
[5:49:35] audio and music analysis. Right? Because we are doing speech-to-text, so we need this. So, it provides various functions to quickly provides various functions to quickly extract key audio features and metrics
[5:49:47] from the audio files. Okay, and this librosa can also be used to analyze and manipulate audio files in a variety of formats such as a variety of formats such as WAV, MP3, M4A, and like that.
[5:50:02] Okay. So next, we will import torch. I hope everyone know torch. torch is nothing PyTorch. It is a machine learning library based
[5:50:16] on the torch library used for applications such as computer vision and NLP. And originally, it was developed by the Meta AI and now the part of Linux Foundation, you know, umbrella.
[5:50:29] Foundation, you know, umbrella. Let's import one more thing. Import Let's import one more thing. Import So now, let's import
[5:50:45] Capital. IPython. dot So, now you will be wondering what is this IPython display.
[5:50:58] So, it is an interactive command line for the terminal for Python. And it you know, it can provide a IPython terminal and the web-based notebook platform for Python computing.
[5:51:14] And this IPython have you know, more advanced features than the Python standard interpreter and we can quickly execute a single line of Python code. Nothing. Okay, for that I'm using this.
[5:51:28] Okay, for that I'm using this. And the next is from transformers And the next is from transformers import WAV2 WAV2
[5:51:49] And WAV2 WAV2 WAV2 WAV2 WAV2 WAV2 Tokenizer.
[5:52:08] the notebook on how to fine-tune a speech recognition model. Okay. And this tokenizer, so tokenizer is nothing Tokenization is a conversion
[5:52:20] of a text into meaningful lexical tokens belonging to categories defined belonging to categories defined a lexical program. In case of NLP, so those categories include nouns, verbs, adjectives, punctuation, etc. Okay, now
[5:52:35] adjectives, punctuation, etc. Okay, now and the last library let's import numpy and the last library let's import numpy as np. I hope everyone know numpy, what library used for working with arrays. It also
[5:52:48] It also has a function, you know, for working in domain like linear algebra, Fourier transform, and matrices, and many others. Fine. I hope everyone know about these library, which we have
[5:53:02] Okay, so let's move forward by tokenizer.
[5:53:15] Google do Google me Google do Google do tokenizer.from
[5:53:32] just write just some words and it will show the suggestions. You know, Facebook So, this is again one pre-trained model, okay?
[5:53:46] model, okay? So, we are just importing it. This is nothing, the name. Okay, then model equals to
[5:54:05] for CTC .from pre-trained then Facebook
[5:54:21] vectorizer do do then base
[5:54:34] Now, let's run it. So, now you can see Okay, they are they are downloading.
[5:54:48] So, now let's load the audio file. So, here I will write audio So, here I will write audio on my sampling
[5:55:02] librosa dot dot load So, you know about librosa, right? Then
[5:55:15] I have v. m4a So, we I have already one s- speech or one audio, you can say. speech or one audio, you can say. So,
[5:55:31] the final output, I will you know 16,000 Okay.
[5:55:46] Okay. Already loaded, I guess. Okay. Okay, no issues. I will load again.
[5:56:04] File is loaded. I will rename it to v. Yeah. Fine. So, now it won't the error won't come.
[5:56:20] We have v.m4a. Okay. So, now we have to do
[5:56:35] Let me try it. We have v.mp4a, We have v.mp4a, but I don't know why it is not coming.
[5:56:50] Okay, let me copy the path. Now it will run, I guess. Yeah. So now I will write here audio
[5:57:02] {comma} sampling rate. Okay. Now listen carefully. Then I will write display.
[5:57:23] path {comma}
[5:57:36] true. Okay. message. >> I guess you heard, but again let me play
[5:57:48] >> Hello and welcome. This is an AI voice message. >> Okay. So it is saying hello and welcome. This is an AI voice message. Fine. So now what I will do, I will input some
[5:58:03] So now what I will do, I will input some values. Input values. Input values {equals} to values {equals} to tokenizer
[5:58:15] audio {comma} return tensors {equals} to PT.
[5:58:27] dot input values. Okay. Then here input
[5:58:39] Okay. Then here input values. Okay. Input values. Okay.
[5:58:51] Yeah. So now logits will come here. So we have to now store the logits, which means non-normalized predictions. means non-normalized predictions. Okay. So logits equals to
[5:59:04] Okay. So logits equals to model values. Dot logits. Okay. Then here I will write logits.
[5:59:25] So now what we will do, we will store predicted IDs, then we will pass the logits to values to softmax to get the predictive value. predictive value. Okay. So here I will write predicted
[5:59:43] IDs equals to Okay. Then logits. Comma dimension
[5:59:55] Comma dimension equals to minus one. prediction uh to the tokenizer decode to the transcription.
[6:00:07] Okay. So here I will write transcriptions equals to tokenizer.
[6:00:24] Dot decode. IDs zero.
[6:00:36] So, now let's see our output transcriptions. So, now you can see "Hello and welcome. This is an AI voice voice message." So, >> Hello and welcome. This is an AI voice
[6:00:51] >> Amazing, right? So, this is how you can recognition. It's a very small code line of code, as you can see. Okay. So, now let's move forward and do
[6:01:07] the sentiment analysis. Okay? Or text generation. Okay? Or text generation. So, let me open new drive. So, yeah. So, now I will write it here.
[6:01:22] I will write sentiment {slash} text generation
[6:01:35] Okay? So, first of all, let's import the So, first of all, let's import the files. So, first I will write import import warnings. Then warnings
[6:01:51] dot filter warnings. Okay? Okay? I will write here ignore. NumPy. You already know what is NumPy
[6:02:06] and import Pandas. So, these are some basic Python library and I hope everyone knows. And import Matplotlib for the plots.
[6:02:19] Matplotlib. dot pyplot as plt. as plt. Then I will import seaborn for the
[6:02:32] graphs. The static things. Okay. As soon as that sns.set
[6:02:44] So now we'll import sklearn.model_selection. probably the most useful library for machine learning Python. So So this library contains a lot of efficient tools for machine learning and
[6:03:00] you know statistical modeling including classification you can do regression you clustering you can do from this sklearn. Okay. From
[6:03:12] dot model selection
[6:03:25] test split from sklearn.
[6:03:39] metrics import import f1_score comma confusion_matrix
[6:04:04] So again we'll import things from transformers. pipeline. Everyone know what is this. Then import
[6:04:23] torch. Okay. Now let's run it. Terminate existing to continue if you are interested in using our more session. Okay. Wait.
[6:04:36] Fine now. Which one is open, bro? One minute this. Terminate this. Terminate this. No.
[6:04:48] Okay. Fine. So now let's do sentiment analysis. Okay. The hugging face library provides, you
[6:05:03] know, convenient pipeline as you already know function that allows us to easily perform sentiment on the text. So now let's first import the necessary dependencies
[6:05:15] and create a sentiments pipeline using this line. So I will write here classifier equals to equals to pipeline
[6:05:31] sentiment analysis. type classifier.
[6:05:44] So it will download the pre-trained sentiment analysis pipeline. Pipeline is not defined. Okay.
[6:05:57] Okay. Okay, error is there. Mhm. Yeah. So now we can uh pass the single sentence or a list of sentence to the classifier and now and
[6:06:13] get the predictive sentiment labels and associate Fine. So now just for the testing classifier. So let's write this
[6:06:29] is a great So now you can see here label positive. So label positive typically refers to
[6:06:46] the outcome or a class of interest that the model is designed to predict. So here we are just checking the sentiment analysis model. Okay. So this is why I wrote this.
[6:07:00] Okay. So let's check one more.
[6:07:16] great course. I did not
[6:07:28] understood any of it. Now let's check this. Yeah, perfect. And the score you can see the accuracy score 99% 99% which is
[6:07:43] almost close to 100% and which is amazing. So now you have access to a GPU so you can also utilize this for the faster processing by specifying device using the parameter. Okay. So now
[6:07:59] what I will do first I will Yeah. Fine. So, now I will import data set.
[6:08:15] Fine. So, now I will import data set. Airline tweets. pd means pandas library here we are using.
[6:08:27] Okay, now first import I will import here itself. here itself. So, I have twitter_tweets.csv.
[6:08:42] .csv Let it upload. Let it upload. Yeah, done. So, now let me Yeah, done. So, now let me airline
[6:08:58] tweets .head Why dot head? Means you it will show me the top five rows. Okay, 0 1 2 3 4 5. Okay, you can see tweet ID, airline
[6:09:12] sentiment, neutral, positive, neutral, negative, then this is this. Okay. So, now let's do something. Okay. So, what I will do df equal df means data frame airline
[6:09:32] tweets Then I will write airline sentiment We have airline sentiment this column. comma text
[6:09:45] I need these two columns. Again, df.head So, airline airline sentiment is a neutral and this is what this text is
[6:10:01] all about. Okay? Because these two are the main things, text and the sentiment. So, now let's make plot. So, now let's make plot. Count plot.
[6:10:19] x equals to airline sentiment.
[6:10:51] plt.y label
[6:11:08] draw one graph. Okay? It will show neutral positive sentiment and the negative. So, as you can see the
[6:11:23] negative sentences or the sentiments are the more. So, now So, we have now three classes which do not match the two class available in the hugging face pipeline. So, therefore, we will filter out all
[6:11:39] the rows which have been labeled as neutral. So, df equals to df and again df airline sentiment.
[6:11:55] Fine. was not equals to was not equals to neutral. Okay. Then DF
[6:12:10] target equals to DF airline sentiment uh map
[6:12:27] Then I will write here positive. Negative will be
[6:12:43] Okay. Then print
[6:13:09] 11,541. Okay. So now I will add predictions.
[6:13:22] Okay. Predictions is not defined. Okay. So now what I will do text equals to DF
[6:13:46] dot to list. Then here I will write predictions equals to classifier
[6:13:59] So, here write probabilities equals to prediction then score
[6:14:13] score if predictions starts with
[6:14:32] Okay. else 1 minus prediction score prediction score prediction
[6:14:46] Okay. Then I will write for prediction and prediction prediction Why it is not running?
[6:14:58] This is taking time, too much time. I don't know why. So, as you can see finally we have the output our predictions values. So, this depends on you know, system to system. My system took almost 17 minutes 40 47
[6:15:15] seconds to complete. Okay. So, now let's run it. Okay. Predictions is not defined. Oh, no. predictions Yeah. So, now I will write here
[6:15:31] predictions equals to NP NP dot array NP means numpy pi dot array dot array NP means numpy pi dot array Then I will write one if prediction
[6:15:47] label is positive dot starts with Okay, else
[6:16:00] Okay, else zero prediction and prediction values. and prediction values. Yep.
[6:16:16] accuracy of our model. Print accuracy. Okay.
[6:16:34] Then, I will do round up the values. So, here I'll write np dot the values. So, here I'll write np dot mean. then the target right.
[6:16:49] Yeah. Then, I will write equals equals to predictions. Then, I will write into 100 comma two
[6:17:04] two then I will write percentage. Fine, looks good. Yes. So, as you can see, our accuracy is
[6:17:17] 88.99%, which is, you know, very good again. which is, you know, very good again. So, now let's do some confusion matrix. So, now let's do some confusion matrix. Confusion
[6:17:32] matrix then df target comma, predictions, comma, normalize
[6:17:44] equals to true. Then I will plot I will plot confusion matrix.
[6:18:15] normalize equals to true. Okay, okay, my bad, my bad, my bad.
[6:18:38] labels. Okay. So here uh I will, you know, plot a confusion matrix using seaborn. Okay, so here I will use a r g s which
[6:18:50] is confusion matrix n p dot n array. So which is, you know, labels list. So I which is, you know, labels list. So I will write p l t dot figure
[6:19:04] equals to eight comma six. Plus s n s dot set set font scale
[6:19:20] equals to one point four. Okay. Then let's create the heat map.
[6:19:35] Confusion matrix. comma and not will be true. equals to T. Then confusion map
[6:19:51] Then confusion map equals to equals to labels. Then Y
[6:20:11] ticks labels equals to labels. Okay.
[6:20:24] Yes, so now I will write PLT dot title will be confusion matrix. Then PLT dot X label will be
[6:20:43] will be actual values. Then PLT dot actual values. Then PLT dot show.
[6:20:55] Okay. Why the chart is not coming?
[6:21:25] positive. Okay, now the chart will come.
[6:21:37] So now you can see here this is in the actual and this is a confusion matrix and the negative and the positive ratio Okay. So now let's print the
[6:21:51] uh now let's check the ROC AUC score. So I will write here print ROC AUC score.
[6:22:13] Then here I will write ROC AUC score. Then DF comma probs. probs. Okay.
[6:22:30] what is this ROC AUC score. So this is the area under the ROC curve. So what it does, it sum up how well a model can produce relative scores to discriminate between positive and the negative instance across all the classification
[6:22:44] threshold. So with this ROC score of 94 0.94, which is 94%, we can conclude that the that our pre-trained sentiment analysis model
[6:22:56] and effectiveness so in predicting the sentiment labels. capable of accuracy classifying text into positive or the negative sentiment
[6:23:08] categories. Okay. So now we'll do text generation. Okay, so text generation involves generating creative and coherent thing. So I will write here text.
[6:23:23] Okay. So, text generation involves generating creative and coherent text based on a given prompt or starting point. So, what necessary dependencies and load the data data set of the poem.
[6:23:37] Okay. So, here I'm write poems. Let me write poems equals to pd.read .csv. Don't worry, I will give you these files. Okay.
[6:23:55] So, here I'll I'll write Robert Frost.
[6:24:08] And then I will write poems .head .head five. Or head, just have to write head. Module pandas has
[6:24:22] Module pandas has Okay, not dot underscore csv. Okay, it will show the top five. Stopping by woods on a snowy evening, fire and ice, the aim was song, collection, content, year of
[6:24:38] collection, content, year of publication. Okay. publication. Okay. So, now what we will do, we will write equals to poems
[6:25:01] content .drop NA to to list.
[6:25:13] So, to generate text, we extract individual lines from the poems and use the pipeline text generation function to create a text generation pipeline. Fine? So, here I will write lines equals to
[6:25:28] Then I will write for poem in poem in content in poem dot
[6:25:43] split into this next line. into this next line. Okay?
[6:25:56] Then I will write line dot write a strip. To the right it will add. To the right it will add. Okay?
[6:26:11] So, now lines equals to line for line in lines
[6:26:24] of line then show me the lines.
[6:26:38] Okay. So, now you see as seen here whose woods these are I think I know. Then the next line is ours is in the village though. Then the next line he will not though. Then the next line he will not sing. Okay? Like this. So, now let's
[6:26:52] you know, import that pipeline text generation module. Okay? Gen equals to
[6:27:05] model. You already know You already know generation.
[6:27:18] Okay. So now I will write here lines. Yeah, done. So in line zero you have we have whose
[6:27:32] words these are I think I know. Okay, so now we can and a specific find the parameters such as max length and num return sequence. Okay, why because we have imported this
[6:27:45] text generation module. Okay, for example, see gen gen line Okay, sorry. lines
[6:27:57] zero dot max length generate the this to the maximum 20.
[6:28:13] till 20 word maximum length. Okay, check. Okay, uh Okay, gen expression cannot contain
[6:28:25] perhaps double equals to Well, okay. Now I will run. So see, the line was this much only. Whose words these are I think I know. Whose words
[6:28:41] But here the generated text I wish to go to church because I feel like. Okay, this is how you can do you know text generation. Now let's check for the more.
[6:28:55] gen lines one lines one Okay, then max comma num
[6:29:10] return sequence cells is equal to two. Okay. So, our first line was this, his house is in the village though. Okay.
[6:29:27] 0 1 2 3 4 5 Okay. So, here you can see. See, his house in the village though how well you might say that the place was the same with the This is this is this. So, these are the generated text.
[6:29:40] Okay, and this is another you know, return sequence second. Okay. So, now So, now let me, you know, import
[6:29:54] let me, you know, import text app. creating function wrap X X I don't need this.
[6:30:20] dot fill X comma replace
[6:30:34] white space is equal to false.
[6:30:47] sentence endings is equal to true. Okay, so now out out is equal to generated lines.
[6:31:03] Zero to maximum length. Then print
[6:31:21] to zero. Then Okay. So, now we are setting the bad tokens to EOS tokens, okay? So,
[6:31:38] whose whose these are I think I know. And this is the maximum 30 till 30. So, now I will write here. Okay, preview equals to Okay.
[6:31:53] Okay. So, now what you can do, you can you know, generate a prompt to generate text on a specific topic like this. text on a specific topic like this. Prompt equals to
[6:32:09] have a wide variety variety of applications
[6:32:23] Okay. So, this is my prompt. Okay, this is I'm not importing from the data set. Okay. So, I will write out equals to gen
[6:32:36] gen prompt, {comma} max length
[6:32:53] the output out. Okay, here I will write Anyways, here I will write generated
[6:33:18] So, it is running. Let's wait. Okay, some error is there.
[6:33:51] words. I wrote I wrote 50. So, let me do it again. 100.
[6:34:03] So, now you can see we have we can generate text using So, we have covered, you know, three topics First thing is speech to text using hugging face. Second thing is sentiment
[6:34:20] And the third is text generation using hugging face. And in this text generation we have we did with two methods. One with data set
[6:34:32] and second thing with the you know, by giving prompt like in ChatGPT, you can see. >> On July 25th, OpenAI introduced ChatGPT, a new search tool changing how we find information online.
[6:34:46] Unlike traditional search engines, which require you to type in specific keywords, ChatGPT lets you ask question in natural everyday language just like having a conversation. So, this is a big shift from how we were used to searching
[6:35:00] the web. Instead of thinking in keywords and hoping to find the right result, you can ask now Search GPT exactly what you want to know and it will understand the context and give you direct answers. It's designed to make searching easier
[6:35:15] and more intuitive without going through links and pages. But, with this new way of searching, so there are some important questions to consider. Can Search GPT compete with Google, the search giant we all know? What makes
[6:35:29] Search GPT different from AI overviews, another recent search tool? And how does it compare to ChatGPT, Open AI's popular conversational AI? So, in this video we are going to explore these
[6:35:42] questions and more. We will look at what makes Search GPT special, how it compares to other tools, and why it might change the way we search for information. Whether you are new into tech or just curious, this video will
[6:35:54] break it down in simple words. Stick around to learn more about Search GPT. So, without any further ado, let's get started. So, what is Search GPT? Search GPT is a new search engine prototype developed by Open AI designed
[6:36:07] to enhance the way we search for information using AI. Unlike a typical chatbot like ChatGPT, Search GPT isn't just about having a conversation. It's focused on improving the search experience with some key features. The
[6:36:21] first one is direct answer. Instead of simply showing you a list of links, Search GPT delivers direct answer to your question. For example, if you ask, cancellation headphone in 2024?" Search GPT will summarize the top choices
[6:36:35] highlighting their pros and cons based on expert reviews and user opinions. traditional search engines that typically provide a list of links leading to various articles or videos. The second one is relevant sources.
[6:36:51] Search GPT responses come with clear citations and links to the original sources, ensuring transparency and accuracy. So, this way you can easily verify the information and delve deeper into the topic if you want. The third
[6:37:04] one, conversational search. Search GPT allows you to have a back-and-forth dialogue with the search engine. You can ask follow-up questions or refine your original query based on the responsive you receive, making your search
[6:37:16] experience more interactive and personalized. Now, let's jump into the next topic, which is Search GPT versus Google. So, Search GPT is being talked about a major competitor to Google in the future. So, let's break down how
[6:37:30] they differ in their approach to search. The first one is conversational versus keyword-based search. Search GPT uses a conversational interface, allowing user to ask question in natural language and refine their queries through follow-up
[6:37:45] questions. So, this creates a more interactive search experience. On the keyword-based search, where a user enter pages. The second thing is direct answer versus
[6:37:58] list of links. So, one of the Search GPT's standout feature is its ability to provide direct answers to the question. It summarizes information from the various sources and clearly cites them. So, you don't have to click through
[6:38:11] Google typically present a list of links, leaving user to sift through the need. The third one, AI-powered understanding versus keyword matching.
[6:38:23] Search GPT uses AI to understand the intent behind your question, offering more relevant result even if your query isn't perfectly worded. Google's primary method is keyword matching, which can sometimes lead to less accurate result,
[6:38:37] especially for complex queries. The fourth one, dynamic context versus isolated searches. So, Search GPT maintains content across multiple interaction, allowing for more personalized responses.
[6:38:49] Whereas Google treats each search as a separate query without remembering previous interaction. And the last one, real-time information versus index web pages. Search GPT's aim to provide the latest information using real-time data
[6:39:02] from the web. Whereas Google vast index is comprehensive but may include outdated or less relevant information. So, now let's jump into the next topic, which is Search GPT versus AI overviews. So, Search GPT and AI overviews both use
[6:39:17] AI, but they approach search and information delivery differently. It's also worth noting that both tools are still being developed, so their features and capabilities may evolve and even overlap as they grow. So, here are the
[6:39:29] differences. The first one is source attribution. Search GPT provides clear and direct citation link to the original sources, making it easy for user to verify the information. Whereas AI overviews include links, the citation
[6:39:41] may not always be clear or directly associated with specific claims. The second one is transparency control. Search GPT promises greater transparency by offering publishers control over how their content is used, including the
[6:39:56] option to opt out of AI training. AI overviews offer less transparency regarding the selection of content and the summarization process used. The next one is scope and depth. Search GPT strives to deliver detailed
[6:40:10] and comprehensive answers pulling from a broad range of sources, including potential multimedia content. And in AI overviews offers a concise summary of key points, often with links for further exploration but with a more limited
[6:40:24] scope. Now, let's jump into the next part. Search GPT versus ChatGPT. Search GPT and ChatGPT, both developed by OpenAI, share some core features but serve different purposes. So, here are some
[6:40:37] differences. The first one is primary purpose. Search GPT designed for search, the web. Whereas Chat GPT focus on conversational AI, generating text responses. The second one is information sources.
[6:40:52] Search GPT relies on real-time information from the web. Whereas Chat data, which might not be current. The third one is response format. Search GPT prioritize concise answers with citation and source links. So,
[6:41:06] generating longer text, summarizes, creative content, code, and etc. The next feature is use cases. Search GPT ideal for fact-finding, research, and task requiring up-to-date information. Whereas Chat GPT is
[6:41:21] suitable for creative writing, brainstorming, drafting emails, and other open-ended tasks. So, now question arises, when will Search GPT is currently in a limited
[6:41:33] prototype phase, meaning it's not yet widely available. Open AI is testing and improve the tool. So, if you are interested in trying Search GPT, so you can join the waitlist on its web page, but you will need a
[6:41:47] Chat GPT account. A full public release by the end of 2024 is unlikely as Open AI hasn't set a timeline. It's more probable that Search GPT features will gradually added to the Chat GPT in 2024 or in 2025 with a potential standalone
[6:42:03] release later based on testing and the feedback. So, with this we have come to question or doubt, please feel free to ask in the comment section below. Our possible. >> Let's start with a simple question. Why
[6:42:16] should we use LangChain? Imagine you're working with large language models like GPT-4 or Hugging Face models, and you want to take their capabilities further, like integrating them with your own data sources or allowing them to take action
[6:42:29] based on information they retrieve. This is where LangChain comes in. LangChain allows you to build intelligent applications by connecting large sources. It can turn static AI interactions into
[6:42:44] dynamic data-aware workflows. One of the best parts is you don't have to manually code everything from scratch. LangChain abstracts away much more complexity of working with LLMs, allowing developers to focus on building functional
[6:42:58] applications instead of wrangling API calls and managing data pipelines. So, LangChain is set to play even bigger role in AI development because it enables you to harness true power of generative AI by connecting it with
[6:43:11] real-time data and external tools. So, now we have understood what LangChain is. Let us now understand how to install LangChain. So, let's start with the installation of LangChain. Uh we'll just simply go to the website
[6:43:23] and we'll just simply go to the website docs part and we'll just read through this documentation. So, here it has explained what LangChain is and what are
[6:43:35] the framework consisting of. So, we also have these tutorials on how do we install LangChain. Okay. So, for installing you can just simply click on this quick start and uh see here it has written how do we set up on Jupyter
[6:43:51] notebook. So, this is the command if you want to install LangChain. We will use the PIP command. So, just simply you can copy this command PIP install LangChain and you have to open your command prompt or the terminal in your computer and
[6:44:07] here you have to simply copy-paste the command. So, as you can see it has it will load all the packages which is required for installing LangChain. So, you can see here requirement already satisfied. This is because I had already
[6:44:22] satisfied. This is because I had already installed my LangChain before. So, uh we have understood how to install this LangChain by using this command and you can also install the LM chain. We'll understand it later. Let me just show
[6:44:35] First, we have understood this LangChain, then we have the Pinecone client. So, we'll just simply search here Pinecone client
[6:44:47] and it will redirect us to this page. So, Pinecone client is actually a vector store for storing storing and retrieving embeddings, which we will use in the later steps. So, Pinecone is also used
[6:45:01] to actually, you know, create secret API keys. So, here you can also create the API keys. You can also read the documentation part. So, so we'll understand how to create API
[6:45:14] keys using OpenAI, but first, let us install Pinecone client in our system. So, we'll again go to command prompt this terminal and we'll just copy-paste pip install Pinecone client. So, you can see here it will download and install
[6:45:30] all the packages required. So, it has been installed. Now, the third thing we're talking about is OpenAI client. So, we'll use OpenAI models for a So, we'll use OpenAI models for a language large language task. So,
[6:45:44] so we'll just simply search here OpenAI, okay? And so, it has redirected us to this page, OpenAI platform. And uh Okay,
[6:45:57] before starting this, so this is the platform here where you so this is the platform here where you can create, export an API key uh in OpenAI, okay? So, you can see here overview, quick start, concept,
[6:46:10] everything is there. And uh To create an API key, we'll just simply click here and here you have to select this option, create a new secret key. Suppose I give my secret key name, anything you can
[6:46:24] give. Suppose uh I give test test 1 2 3, okay? And permissions is all and we create the secret key. Now, uh you need to uh actually save your key. We'll just copy this key
[6:46:38] because it will be required later while uh debugging the code. So, we will just copy-paste this secret key. We will require it later and then done. So, these are the keys I have created. So, actually ChatGPT and other LLM models
[6:46:54] like OpenAI and Hugging Face uses LangChain to integrate with other APIs to create your own custom LLM models or chatbots. So, suppose here we have uh logged in our ChatGPT here and uh if I
[6:47:08] search here who won the uh who won the uh WTC World Cup
[6:47:21] So, here it has shown the answer, but for example, uh if I search for example, uh if I search who won the cricket who won the cricket match World Cup.
[6:47:40] given answer as "As my last knowledge updated in April, men's World Cup has not taken place yet." This is happening because the uh this ChatGPT uh older version has not been trained on the latest uh upcoming news or whatever the
[6:47:54] new technology is. So, by using LangChain, you can integrate with other APIs and you can create your own customized LLM models or chatbots, which help you to train your own custom data using various tool and
[6:48:07] APIs. So, uh before we move on, I've already showed you how to create the secret uh API key and how you have to store that uh API key address. So, first we have already understood how do we install
[6:48:22] LangChain here. So, by using the pip command we do it. And command we do it. And So, you also need to install Python 3.8 or later installed in your system. So, I already have Python installed. And to
[6:48:35] check that, you can just simply I already have it installed in my system. here Python minus minus version. And click enter. So, as you can see it has shown me the Python version which is
[6:48:50] installed in my computer. This So, the second step is already we have discussed which is OpenAI API key here. So, second step we have already API key. So, we have to sign up in our
[6:49:04] OpenAI, then go to the API key section, and then create a new secret key. And these are the keys I have created, and you can just keep it later later you'll use that. So, now we'll come to the third step which is create a project
[6:49:18] third step which is create a project directory and setup. So, what we do is we have Jupyter installed in our system. So, we'll just go to command prompt here and type j u p y Jupyter notebook.
[6:49:34] So, it will redirect us to the Jupyter notebook installed in our system. So, if not you can just simply go from here. wait. Now, you just need to click on this
[6:49:48] new and Python 3 I I kernel because Python has been installed in my system. here. And here you can just give the prompts the command. Before that, you have to create a Python file which we can also
[6:50:03] create this Python file in Visual Studio Code. Just simply go to Visual Studio Code here and just simply click on file, new file
[6:50:15] and just simply click on file, new file and I'll just type your python.py. And you can you have to first create store the API key. For this we use the open AI
[6:50:28] underscore API underscore key equal to equal to and just give your secret key.
[6:50:48] your secret key and just store this. So this ensures that your API key is stored securely and it can be used whenever needed. Now the step four is to initialize the project and install the required libraries. So you need to add
[6:51:00] some additional libraries like Streamlit to make a user interface. So let's add that to in our project folder. You can either create a text file named requirements.txt and then install all these. We have already
[6:51:14] installed the open AI LangChain. We just need to install Streamlit. So here you need to install Streamlit. So here you can just give the command pip install Streamlit. So as you can see this I have already installed Streamlit before. Same
[6:51:31] wise you can also install open AI if it is not installed in your system using the command terminal in Windows PowerShell. So we have understood this how what all packages and what all we need to
[6:51:43] install. Now the next step is to build your first LangChain app. To create a simple app which uses input query and the app will generate a response using open AI's GPT model. So you have to create a Python file named
[6:51:57] main.py here. So So as you can see I've already imported this main.py and this is my code here. Import Streamlit as ST from LangChain and the constants I've created and then I have initialized the OpenAI
[6:52:13] with our API key. So, you have to you have to just type this prompt. Here, I'm using VS Code here. You can also do it in your Jupyter notebook. And then to create the Streamlit app, you have to give a title, LangChain demo with
[6:52:26] OpenAI. So, this is the title I have provided. And then the text input prompt for prompt, the prompt is st.text_input and enter the prompt. You can just type enter the prompt or whatever you wish to. And then display the response. So,
[6:52:40] if prompt response is llm.predict prompt, so you can use the predict method for llm. So, here what the app will So, after creating and debugging this in the terminal, so your app will initialize OpenAI using your API key and
[6:52:55] the user inputs a prompt through the Streamlit interface. LangChain processes the input and sends it to the OpenAI's GPT model and the AI generates a response which is then displayed in the app. So, now you can use all these
[6:53:08] prompts to run on your app. So, to do this, you can just to see your app in and run the following command which is streamlit run main.py. So, you can just simply go to the terminal here and just simply type
[6:53:22] uh simply type the command which is stream run main.py. your browser will open displaying the app and you can also type any question
[6:53:37] to the input box. So, last, now we have understood all these steps. So, this was a quite basic tutorial on how to install LangChain and then, you know, integrate it with the app. You can also customize and expand. So, LangChain's flexibility
[6:53:52] allows you to integrate other APIs also, external data sources, or even add memory to your AI application. So, whether you're building a simple chatbot or a more complex AI system, the possibilities are endless. So, by
[6:54:04] following all these steps, you will have a fully functioning app running in your system in no time. What if I have told you I've tested over 1,000 AI tools? Yep, after all that exploring, experimenting, and a few late nights,
[6:54:20] I've finally narrowed them down to absolutely the best. In this video, I'm absolutely the best. In this video, I'm going to share my top picks of 10 tools from each of the five biggest AI categories that are actually worth your
[6:54:33] time. Whether you're into image creation, writing, voice tools, video generation, or AI coding assistants, I have got something lined up for you. These aren't random tools that kind of work. These are the ones that stood out,
[6:54:48] the ones that genuinely get the job done. So, if you have ever felt overwhelmed by the flood of AI tools out there, this video is going to be your cheat sheet to the top of all commerce in every category. Let's start with the
[6:55:02] category of image creation. In the first place, we have Midjourney. If you're after breathtaking artistic visuals, this one is for you. It's especially great for concept art, fantasy scenes, or anything with surreal polished look.
[6:55:19] or anything with surreal polished look. Next is DALL-E 3, integrated right inside the ChatGPT. What makes this tool stand out is how well it understand prompts. It nails small details, and you can even edit parts of your image
[6:55:34] directly. Super useful for content creators or marketers who need fast, accurate visuals. Third is Stable Diffusion, and it's go-to for anyone who wants full control. It's open source, so you can run it locally, train your own
[6:55:50] models, and even build your own styles. If you're technically or just love to tinker, this one is for you. Coming to number four is Leonardo AI.
[6:56:02] It's made with game developers and storytellers in mind. It has pre-trained models for characters, environments, and even items. So, if you are building a
[6:56:15] world, this tool makes it faster and easy to bring it to life. At number five, we have Adobe Firefly. This one's perfect for professionals. It fits right into the Adobe ecosystem if you're already using the Photoshop or
[6:56:31] Illustrator. It's a smooth experience, the image quality is high, and text effects are great for branding or designing work. Number six is Playground
[6:56:43] AI. This is probably the most flexible tool on this list. It's web-based, fast, and great for trying out multiple styles without any complexity. If you want something lightweight but powerful, definitely check this out. Art Breeder
[6:56:58] comes in at number seven, and it's a bit different from others. Instead of generating images from prompts, you mix and evolve existing images. Great for character design, especially faces. It's
[6:57:13] super collaborative and creative, though a bit limited in image resolution. Number eight is Nightcafe. This one's great if you want variety. It supports multiple generation styles like VQGAN and Stable Diffusion. There is also a
[6:57:30] strong community aspect, so you can share your work and get inspired by share your work and get inspired by others. At number nine, we have Fotor. It's clean, simple, and ideal for creating social media graphics, posters,
[6:57:43] or marketing visuals. You don't need any design skills. It's perfect for your quick results. And finally, number 10 is deep dream generator. This one's all about the weird and abstract. It creates psychical
[6:57:59] dream-like visuals that are super unique. It won't fit every project, but experimental or surreal work, it's awesome. Next category is voice generation. These AI tools can narrate, speak, or even clone your voice with
[6:58:15] shockingly realism. Whether you're creating YouTube videos, podcasts, audiobooks, or building characters for games, these are the voice tools that actually deliver. In the first place, we have ElevenLabs.
[6:58:29] It's the most natural-sounding AI voice out there. You get emotion, depth, and even voice cloning that sounds incredibly real. Coming to the second is Murf AI. This one is great for polished voiceover, perfect for videos,
[6:58:45] presentation, and training content. It even includes background music and timing control. At number three, we have got LOVO AI, also known as Genny. With a
[6:58:57] huge voice library and support for 100 plus languages, it's ideal for creators who want expensive, multilingual voiceovers. In the fourth place, we have play.hd. It's fast, reliable, and offers detailed
[6:59:12] control over the tone, pitch, and speed. Great for podcast, articles, or any voice first content. Number five goes to Resemble AI. This one's all about
[6:59:24] real-time voice cloning and emotionally delivering. Perfect for interactive content like games and virtual assistants. At number six, we have Descript Overdub. Audio by editing text. It's a lifesaver for podcasters and
[6:59:40] video creators fixing mistakes or adding new lines. Seventh on this list is Typecast. It combines realistic voice AI with animated characters, making it perfect for explainer videos, YouTubers, and
[6:59:55] storytelling content. At number eight, we have Speechelo. Originally made for reading, but now you can use it to turn any text into clean, human-like speech.
[7:00:07] Great for audiobooks and productivity. Number nine is Synthesys, a solid choice for business voice over, whether it's ads, explainers, or training videos.
[7:00:19] This one gets it done with professional sounds. And finally, in the 10th place, we have Podcastle. It's an all-in-one podcasting platform with built-in voice generation, recording, and editing. Perfect for creators on the go. Now,
[7:00:36] let's talk about writing. Whether you're crafting blog posts, emails, scripts, or social media captions, AI writing tools can save you hours and spark ideas you
[7:00:48] wouldn't have thought of. I have tested dozens, and these are 10 that stood out how well they actually help you write smarter and faster. In the first place, smarter and faster. In the first place, we have ChatGPT by OpenAI. It's a gold
[7:01:03] standard. From brainstorming to full-on article drafts, it handles everything with context and style. Especially with GPT-4, it's like having a writing GPT-4, it's like having a writing assistant that just gets it. Second is
[7:01:18] Jasper AI. Built specially for marketers and content creators, Jasper comes with tons of pre-built templates for product description, ad copy, blog intros, you
[7:01:30] name it. It's fast, efficient, and optimized for high-converting content. At number three, we have Rytr. This one's amazing for versatility. It
[7:01:42] does blogs, landing pages, Google Ads, and even long-form content. It also has a chatbot, so it feels more interactive while creating. Fourth is copy.ai. If you need social media post, product
[7:01:58] If you need social media post, product description, or email subject lines in seconds, this tool nails it. It's super intuitive and beginner-friendly, perfect for quick content sprints. In the fifth place, we have Notion AI.
[7:02:13] Built right into Notion, this tool helps you expand, summarize, and brainstorm inside your notes and docs. Great for people who want writing help in their daily workflow without switching tools. Number six is Sudowrite. This one's made
[7:02:29] Number six is Sudowrite. This one's made for creative writers, such as novelists and storytellers. It helps with characters' development, plot twist, and rewriting the scenes. If you write fiction, this is a secret weapon.
[7:02:43] Seventh is Ink Editor. It combines AI writing with SEO optimization. So, if you want your content to rank while sounding human, this tool does both.
[7:02:55] Great for bloggers, content marketers. Coming to number eight is Hypotenuse AI. It is like a smart autocomplete that works across your browser. It helps you draft emails, social posts, and even essays. At number nine, we have Got
[7:03:11] Writer. Clean, fast, and super affordable. Writer is a solid tool for small businesses and freelancers who want quick content without any complexity. And finally, number 10 is Wordtune. It
[7:03:25] is not a full-blown writing platform, but it's amazing at rewriting, rephrasing, and polishing your text. Great for improving tone, clarity, and shortening long sentences. All right. Now, let's move on to into
[7:03:40] the most powerful category, which is AI video creation. Whether you're a content creator, marketer, educator, or just someone who wants to make videos without cameras, editing headaches, these tools can seriously level up your workflow. In
[7:03:56] the first place, we have got Pictory. This tools turns scripts or blog posts into professional-looking videos in minutes. It picks stock footage and add
[7:04:08] minutes. It picks stock footage and add music and even guarantees subtitles all automatically. Next up at number two is Synthesia. It's famous for its AI avatars that look and sounds like real people. Just type in your script, pick
[7:04:24] an avatar, and boom, you have got your studio-style video with zero filming. In the third place is Runway. This one's creative powerhouse. You can remove the video backgrounds, edit scenes, and text commands, and even generate a new video
[7:04:41] content using Gen-2. It's specially great for short-form content or experimental visuals. Coming at number four is InVideo. It's built for marketers, content creators who want to make polished promo videos, reels, or
[7:04:58] even YouTube content. Tons of templates, stock assets, and AI script assistance stock assets, and AI script assistance makes it fast and easy. Fifth is Lumen5. Ideal for turning blog posts or articles into animated videos. You just paste in
[7:05:14] your text, and it automatically adds visuals, animations, and transition to match your message. Great for social media and explained content. At number
[7:05:26] six, we have Descript. While it is more known for podcasting and editing audio. known for podcasting and editing audio. Its video editor is crazy useful. You can edit videos by editing the transcript. It's great for talking head
[7:05:41] content, interviews, and repurposing long-form recordings. Number seven is veed.io, a browser-based editor that is surprisingly powerful. You get subtitles, transition, audio cleanup,
[7:05:56] and more. All with a super clean UI. It's great for social media content and quick edits on the fly. Eighth is rephrase.ai. This one lets you create personalized video messages using AI avatars. Perfect
[7:06:12] for sales, onboarding, or sending custom messages at scale without having to record anything manually. In the ninth place, we have Steve.ai. Super beginner-friendly, it turns simple text into videos using animations and
[7:06:29] voice-over. Great for explain videos, internal training, or educational content. And finally, number 10 is elai.io. It creates AI avatar videos from just a prompt or script. Similar to Synthesia,
[7:06:46] but with a focus on e-learning and corporate training content. Scalable, clean, and no need of production crew. Now, let's get into the most game-changing areas of AI, coding assistance. Whether you are a beginner
[7:07:01] or a pro developer, these tools can help you write better code, debug faster, and even learn new languages or frameworks without getting overwhelmed. In the
[7:07:13] first place, we have GitHub Copilot. Powered by OpenAI, this tool is a autocomplete on steroids. It suggests whole functions, helps you write whole functions, helps you write boilerplate, and even explains code, all
[7:07:28] from inside your editor. If you're coding on a daily basis, this is a must have. Second is Codium. It's a free, fast, and supports a wide range of languages. It works like a co-pilot, but adds some extra customization and a lot
[7:07:44] of devs love it for its speed and clean interface. At number three is Replit Ghostwriter. Especially great for beginners and educator, Replit's AI can
[7:07:56] help you code right inside the browser. No setup needed. It also helps you explain errors and teach you as you build. Fourth is Tabnine. One of the easiest code compilation tool. Tabnine runs locally for privacy and works
[7:08:13] across multiple IDEs. It's great for developers who want predictive coding without sending data to the cloud. In the fifth place, we have Amazon CodeWhisperer. Designed us to integrate smoothly with AWS tools and workflows.
[7:08:29] It's perfect if you're building cloud-native application or working on AWS ecosystem. Coming at number six is Codex by OpenAI. It's a underlying model
[7:08:41] behind co-pilot and if you're working in an experimental or custom environment, it offers a deep integration potential for AI-driven apps. Number seven is Mutable AI. A newer player focused on
[7:08:56] speeding up your workflow with smart suggestions, automatic doc generation, and code refraction. It's great for clean, productive-level code. Eighth is Kite. It is one of the OG AI assistants for Python and still holds up if you
[7:09:13] want lightweight, fast suggestions without overwhelming features. At number nine, we have Sourcegraph Cody. It's made for enterprises terms that work on massive code bases. It gives powerful contextual help and scales well across
[7:09:29] contextual help and scales well across big projects. And finally, number 10 is AskCody. It's super handy for generating snippets, SQL queries, and even API docs. A nice tool for quick task when you don't want to dive too deep. And
[7:09:45] that wraps up the top AI tools across image creation, voice generation, writing, video production, and coding. These tools aren't just cool. They actually help you get things done faster, better, and with less effort. If
[7:10:01] you found this helpful, drop a like, subscribe, and let me know in the comments which tool you're excited to try or if there is anything that you love that I missed. >> So, now what is RNN?
[7:10:14] RNN are a type of neural network that are designed to process sequential data. They can analyze data with temporal dimension such as time series, speech, and text. RNN can do this by using a hidden state
[7:10:28] passed from one time step to the next. The next hidden state is updated at each of the time step based on the input and the previous hidden state. RNN are able to capture short-term dependencies in sequential data, but they struggle with
[7:10:42] capturing long-term dependencies. Why the LSTMs are made. So, moving forward, let's discuss types of LSTM gates. So, LSTM models have three types of gates. The input gate,
[7:10:55] gates. The input gate, the forget gate, and the output gate. So, let's first discuss the input gate. The input gate controls the flow of information into the memory cell deciding what to store.
[7:11:08] The input gate determines which values from the input should be updated in the It uses a sigmoid activation function to scale the values between zero and 1 and then applies point wise multiplication to decide what information to store.
[7:11:22] Next is forget gate. Controls the flow of information out of the memory cell deciding what to discard. should be discarded from the memory cell. It also uses a sigmoid activation
[7:11:36] function to scale the values between 0 and 1 followed by point wise information to forget. The last one is output gate. Controls the flow of information out of the LSTM
[7:11:49] deciding what to use for the output. The output gate determines the output of the LSTM unit. It uses a sigmoid activation function to scale the values from 0 to 1 then applies point wise multiplication to produce the output of
[7:12:02] the LSTM unit. So these gates implemented using sigmoid function are trained using back propagation. They open and close based on the input and the previous hidden state allowing the LSTM to selectively retain or discard
[7:12:15] information effectively capturing long-term dependencies. So now let discuss application of LSTM. LSTM models are highly effective and used in various application including video analysis, analyzing video frames
[7:12:29] to identify action, object and scenes. The second is language simulation. Tasks like language modeling, machine translation and text summarization. The third one is time series prediction. So predicting future values in a time
[7:12:43] The fourth is voice recognition. Tasks such as speech to text Tasks such as speech to text transcription and command recognition. The last one is sentiment analysis. Classifying text sentiment as positive,
[7:12:55] negative or neutral. So there are many more examples of LSTM. So now let's move forward and understand LSTM model and how it works with example. Let's consider the task of predicting the next word in a sentence. This is a common
[7:13:07] application of LSTM networks in natural language processing. So I will break it down a step by step using the analogy of remembering a story and deciding what comes next based on the context. So, imagine you are reading a story. As you
[7:13:20] read, you need to remember what has happened so far to predict what might happen next. So, let's illustrate with a simple example sentence. The cat sat on the dash. So, you want to predict the next word, which could be mat or roof or
[7:13:34] something else. An LSTM network helps us make prediction by remembering important parts of the story and forgetting irrelevant details. So, now let's dive into a step-by-step process. So, step-by-step explanation
[7:13:47] using LSTM. The first one is reading the story, input sequence. As you read each word in the word sentence, you process it and store relevant information. For example, the you understand it's a determiner. Cat, you know it's a noun
[7:14:01] and the subject of the sentence. Sat indicates the action performed by the subject. On, preposition indicating the relationship between the cat and the next noun. So, this sequence diagram showing the words being read and
[7:14:15] processed. So, second comes forget gate. As you move through the sentence, you longer important. For instance, you might decide that knowing the is less important now that you have cat
[7:14:28] and sat. So, the word forget gate helps discard this less important information. So, this sequence diagram you can see on the screen showing how relevant information is discarded.
[7:14:40] So, the third one, input gate. When you read on, you need to decide how relevant this information is. So, this sequence diagram in the screen is showing how new information is integrated with the last one.
[7:14:55] Okay, the fourth one is the cell state, memory part. So, this is like your memory of the story so far. It carries information about the subject cat and the action sat on. So, it updates the new information as you read each word.
[7:15:10] Okay, the cat sat on the retaining the important context. So, this sequence diagram showing how the memory is updated with the new information. So, the last one is output gate. When you need to predict the next word,
[7:15:23] the output gate helps you decide based on the current memory cell state. So, it on the current memory cell state. So, it uses the context the cat sat on the So, the predict the next word might be mad because cat and mad are often
[7:15:36] associated with the context. So, it can predict anything. The cat sat on the table or on the sofa, anything. But, mad Why I'm saying mad? Because cat and mad are often associated in the same context.
[7:15:51] So, this diagram is showing the prediction of the next word based on the current memory. So, there are many applications where you can use LSTM in the sentence. So, by the using of LSTM gates, input
[7:16:06] gate, forget gate, and output gate, and updating the cell state, the network can predict the next word in a sequence by maintaining relevant context and discarding unnecessary information. This step-by-step process allow LSTM network
[7:16:19] to effectively handle sequence and make accurate prediction based on the documentation and you could see the new features introduced with the chat GPT-4o. So, these are the improvements.
[7:16:32] Uh one is the updated and interactive bar graphs or pie charts that you can create. And these are the features that you could see here. You could change the color, you could download it. And what we have is you could update the latest
[7:16:46] and Microsoft OneDrive. And we have the interaction with tables and charts in an new expandable view that I showed you here that is here. You can expand it in the new window. And you
[7:16:59] presentations and documents. Moreover, you can create a presentation also. That we'll see in further. And here we have how data analysis works in ChatGPT. You could directly upload the files from
[7:17:15] Google Drive and Microsoft OneDrive. I will show you guys how we can do that and where this option is. And we can work on tables in real time. And there we have customized presentation, ready charts. That is you
[7:17:29] charts based on a data provided by you. And moreover, a comprehensive security and privacy feature. So, with that guys, we'll move to ChatGPT. And here we have the ChatGPT 4.0
[7:17:44] So, before commencing guys, there's a aspiring data analyst looking for online training and graduating from the best universities or a professional who elicits to switch careers with data
[7:17:57] analytics by learning from the experts, then try giving a shot to Simply Learn's Purdue Post Graduate Program in Data Analytics in collaboration with IBM. You and pin comment. So, let's get started with data analysis
[7:18:11] part. So, this is the pin section or the insert section connect to Google Drive, connect to Microsoft OneDrive, and you can upload it from the computer. This option was already there. That is
[7:18:24] upload from computer. And you can upload at least or at max the 10 files. That Excel files or documents. So, the max limit is 10.
[7:18:36] And if you have connected to Google Drive, I'll show you guys. Drive, I'll show you guys. Uh I'm not connecting you. could upload it from there also. And there's another cool update. That is
[7:18:51] ability to code directly in your chat. Uh so, while chatting with ChatGPT, I'll show you guys how we can do that. And you could find some new changes that is in the layout. So, this is the profile
[7:19:04] section. It used to be at the left bottom, but now it's moved to the top right and making it more accessible than ever. So, let's start with the data analysis part and the first thing we need is
[7:19:18] data. So, you can find it on Kaggle or you could ask ChatGPT 4.0 to provide the data. I'll show you guys. So, this is the Kaggle website. You can sign in here and click on data sets. You can find all
[7:19:32] computer science, education, classification, computer vision. Or else, you could move back to ChatGPT and you could ask the ChatGPT 4.0 model to generate a data and provide it in Excel format. So, we'll ask him
[7:19:48] ask him "Provide a data set "Provide a data set that I can use for data analysis and provide in
[7:20:03] CSV format." So, you could see that it has responded that I can provide a sample data set and he has started generating the data set he has started generating the data set here.
[7:20:17] only 10 rows and he's saying that I will now generate this data set in CSV format. First, he has provided the visual presentation on the screen and now he's generating the CSV format. So, if you want more data, like if you want
[7:20:31] 100 rows or 1,000 rows, you could specify in the prompt and ChatGPT will generate that for you. So, we already have the data. I will import the data. You could import it from here. Or else, you can import it
[7:20:45] from your Google Drive. So, we have a sales data here. We will open it. So, we have the sales data here. So, the first step we need to do is data cleaning. So, this is the crucial step
[7:20:58] to ensure that the accuracy of our analysis is at its best. So, we can do that by handling missing values. That is missing values can distort our analysis. And here ChatGPT-4 can suggest methods to impute these values such as using the
[7:21:12] mean, median, or a sophisticated approach based on data patterns. And handling the missing values, we will remove duplicates and outlier detection. remove duplicates and outlier detection. So, we'll ask ChatGPT
[7:21:26] clean the data if needed. that would be clean the data if needed. And this is also a new feature. You can
[7:21:38] see the visual presentation of the data here that we have 100 rows here and the columns provided that is sales ID, date, product category, quantity, and price per unit, and total sales.
[7:21:52] So, this is also a new feature that Okay. Uh we just head it back. We'll move back to our ChatGPT chat We'll move back to our ChatGPT chat here.
[7:22:15] So, you would see that ChatGPT has cleaned the data and he's provided that it has checked for missing values, checked for duplicates, and ensured checked for duplicates, and ensured consistent formatting. And he's saying
[7:22:28] Okay. Okay. So, now we'll ask him that Okay. So, now we'll ask him that execute
[7:22:40] and provide the clean data. As ChatGPT has provided that these would be the steps to clean the data. And let's see. So, he's provided a new CSV file
[7:22:55] with the clean sales data. We will download it download it and ask him to use the same file only. new cleaned sales data CSV file
[7:23:24] what analysis we can do further. But, once our data is clean, the next step is visualization. So, visualizations help us understand the representation. So, the first thing we will do is
[7:23:37] we will create a prompt for a generating the histograms. And we'll do that for the age distribution part. So, we'll write a prompt that generate a So, we'll write a prompt that generate a histogram.
[7:23:52] Generate a histogram to visualize the distribution of customer ages. Visualize the distribution
[7:24:09] code button. If you just select the text and you would find this reply section, just click on that and you could see that it has selected the text or what you want to
[7:24:21] to get or the prompts started with ChatGPT. make it cross and you could see that it has provided the histogram here. And these are the new features here.
[7:24:35] we could see that he is providing a notification that interactive charts of this type are not yet supported. That is option. I will show you the color change option
[7:24:47] bar chart section. So, these features are also new. You can download the chart from here only. And this is the expand chart. If you click on that, you could see that you could expand the chart here and continue chat
[7:25:00] with ChatGPT here. So, this is the interactive section. the histogram that is showing the age range are from 18 to 70 years with the
[7:25:14] distribution visualized in 15 bins that he has created 15 bins here. And now moving to another visualization that we'll do by sales by region. So, before that I will open the CSV file
[7:25:30] that is provided by the ChatGPT. So, you guys can also see what data he So, you guys can also see what data he has provided.
[7:25:42] you could see that we have columns sales ID, date, product category, quantity, price per item, total sales, region, and sales person. So, now moving back to ChatGPT. So, now we'll create a bar chart showing total
[7:25:55] sales by region. So, we'll enter this prompt that create So, we'll enter this prompt that create a bar chart
[7:26:13] creating bar charts or histogram charts, but we can do that for only two columns. If you want to create these data visualization charts, we need two So, you could see that he has provided the response and created the bar chart
[7:26:27] And this is the interactive section. You could see that there's an option to that, we can't like we are not getting any
[7:26:39] information we scroll on that. And if I enable this option, you could see that I can visually see how many numbers this bar is indicating. And after that we have the change color section.
[7:26:53] provided. So we can change it to any color that is provided here or you could just write the color code here. that is download and another is the expand chart section.
[7:27:10] And if you need uh what code it has done to figure out this bar graph, so this is the code. You could use any ID to do so. You could use any ID to do so. If you don't want the presentations or
[7:27:24] the visualizations of the bar charts here, you could use your ID and use the Python language and it will provide the code for you. Just take your data set and read it through pandas and generate the bar
[7:27:36] through pandas and generate the bar charts. category-wise sales section. So here we'll generate a pie chart showing the proportion of sales for each product category.
[7:27:51] category. So for that we'll write a prompt.
[7:28:17] generating the pie chart. And this is also an interactive section. If you click on that, you would be seeing a static pie chart. And if you want to change the color, you can change for any section that
[7:28:31] would be clothing, electronics, furniture, or kitchen. And similarly, we have the download section and the expand chart section. So, this is how this new ChatGPT 4.0 model is better than ChatGPT 4.
[7:28:45] That you could use a more interactive pie charts. You could change the colors for that. And you can just hold over these bar charts and them. So, after this data visualization, now
[7:29:00] we'll move to statistical analysis. So, this will help us uncover patterns and So, the first thing we'll do is correlation analysis. And for that, we'll write the prompt, "Analyze the correlation between age and purchase
[7:29:15] amount." So, this correlation analysis helps us understand the relationship between two variables. So, this can indicate if older customers tend to spend more or less. So, we'll find out that
[7:29:28] by analyzing the data, and we'll provide a prompt to ChatGPT that analyze the correlation between age and purchase amount.
[7:29:53] You could see a scatter plot that shows the relationship between customer age and total sales. That is with a calculated correlation coefficient of approximately 0.16. So, this indicates a weak positive correlation between age
[7:30:08] and purchase amount, suggesting that as customer age increases, there's a slight well. So, you could just see the scatter plot That if the age increases, so it is not correlated
[7:30:23] to sales, as you could see an empty graph here. graph here. So, till 40 to 50 years of age or the 70 years of age, you could find what amount they have spent here. That
[7:30:36] is the total sales accumulated by these ages. So, now moving to sales trend. So, here we'll perform a time series given dates. So, what does this do is
[7:30:52] time series analysis allows us to examine how sales amount changes over time, helping us identify trends and seasonal patterns. seasonal patterns. So, for that we'll write a prompt.
[7:31:11] time series analysis of purchase amount
[7:31:29] provided us the response, and here's the time series plot showing total sales And each point on the plot represents the total sales for a particular day. So, through this you can find out and
[7:31:44] the businesses find out which is the seasonal part of the year and where to stock up their stocks for these kind of dates. And after that, you could also do customer segmentation. So,
[7:31:58] what does this do is So, we can use clustering here to segment customers amount. So, clustering groups customers into segments based on similarities. This is useful for targeted marketing and
[7:32:12] personalized services. And after that, we have the advanced usage for data analysis. Here we can draw predictive modeling table and do the market basket analysis and perform a customer lifetime value analysis.
[7:32:27] So, we will see one of those and what we'll do is we'll perform a market basket analysis and perform an association rule mining to find frequently bought together products. So,
[7:32:40] the theory behind this is the association rule mining helps identify purchased together aiding in inventory management and cross-selling strategies. So, for that we'll write a prompt that
[7:32:55] find frequently bought together products. So, for that we'll write a prompt here. So, for that we'll write a prompt here. Perform an association rule mining
[7:33:07] to find frequently bought for this prompt what does ChatGPT for a
[7:33:19] this prompt what does ChatGPT for a response to us? But, we don't need a code here. We need the analysis.
[7:33:33] provide code. Do the market basket analysis
[7:33:45] and provide visualizations.
[7:34:03] provided the response that given the limitations in this environment, so he's not able to do the market basket analysis here. So, but he can help us how we can perform this in an ID.
[7:34:17] required libraries, then prepare the data, and here he's providing the So, you could see there are some limitations to ChatGPT 4 also, that he can't do advanced data analysis.
[7:34:32] So, you could use the code in your ID and do the market basket analysis there. So, there are some limitations to ChatGPT 4 also. And now we'll ask ChatGPT, "Can you create a presentation based on
[7:34:46] the data set?" And we'll provide a data set to it also. So, we'll provide a sample sales data.
[7:35:03] "Can you create a presentation or PowerPoint presentation
[7:35:18] and only provide data visualization
[7:35:31] started analyzing the data. And he's stating that and he will start by creating a data visualization from the provided data set and compile them the provided data set and compile them into a PowerPoint presentation.
[7:35:53] provided us the response. And these are all the presentations or the bar graphs that he has created. And now we have downloaded the presentation here. We'll open that.
[7:36:12] that is created by chat GPT-4. >> Before you bring generative AI into the picture, let's think about how data analytics is usually done. Traditionally, if you wanted to know how sales changed from one quarter to
[7:36:26] another, you would need to open your database, then write a SQL query or some pandas code, then generate a chart in Excel or Power BI, and then finally interpret the result by yourself.
[7:36:43] requires strong technical skill. Now, imagine you don't have to do all that. Instead, you just ask a simple question in English. What was the sales growth in quarter two compared to quarter one? A generative AI system will write the SQL
[7:36:58] or Python code for you. And then it is going to run it against the data. And also, it is going to show you the percentage change and even it can give you a short explanation. This means that
[7:37:12] somewhat analytics have become conversational, automated, and predictive. You talk to your data the way you talk to a colleague. Now, you'd be wondering, why does this matter for data analyst? Because it saves time,
[7:37:25] reduces technical barriers, and lets you focus on understanding and acting on mechanics of coding. GenAI is not here to replace analyst, but to act as a co-pilot speeding the
[7:37:40] queries, cleaning data, building visualizations, and even suggesting Now, there are some of the core applications of GenAI in data analytics. insights. So guys, as you can see all over here,
[7:37:55] that traditionally, if you wanted to get an answer from a data, you had to write your own SQL queries or Python code. That means only trained data analyst could do it. But with GenAI, anyone, even a marketing
[7:38:09] manager or HR professional, can simply type a question in plain English. For example, show me a chart of customer churn rate by region. And the AI is going to understand the request and write the code in the background. Run it
[7:38:23] chart. Now, you can imagine how making analytics from the given data has become so much easier. Even when you don't know coding. Next one, if you could see, you are
[7:38:36] going to get automated data storytelling. Numbers on the charts are very useful. But they don't always tell the full story. GenAI goes a step further. It can explain numbers in words. For example,
[7:38:50] instead of just showing a revenue chart, GenAI can automatically write revenue increased 12% in APAC, mainly due to surge in mobile app users. So, you can see how it is saving time for analyst and also it is helping
[7:39:05] non-technical people. Third one is predictive and prescriptive analytics. Most traditional analytics tools tell us what has already happened. But GenAI can
[7:39:17] look at patterns in the data and help predict what might happen next. For example, you can ask something like this, like based on the current trends, underperform in the next quarter? GenAI can not only make the prediction,
[7:39:31] GenAI can not only make the prediction, but also suggest what actions to take to increase the promotion in a particular region. Next is data cleaning and preparation. Every analyst knows the hardest part of analytics is not the
[7:39:43] charts. It's cleaning messy data. values or spelling mistakes or duplicates, then it becomes pretty harder. But GenAI can automatically suggest
[7:39:56] cleaning steps, even generate the code to fix the data set. For example, guys, if customer data has duplicate entries, GenAI can detect and remove them if revenue numbers are missing, and it can
[7:40:09] fill them based on the relative values. This saves hours of manual effort. And finally, you can get visualizations on demand. In traditional BI tools, you
[7:40:21] drag and drop fields, adjust filters, and then format charts. This can take a lot of clicks, but with GenAI, you can just ask, "Create a heat map of sales by product category and region." And boom, you're going to get
[7:40:37] it very much easily. The AI is instantly going to generate a professional-looking chart or even a full dashboard. This makes data visualization faster, easier, and more flexible. So guys, I hope so you would
[7:40:51] have got a brief idea regarding the application of GenAI in data analytics. Now, let us begin to the next part, that is the hands-on tutorial. So guys, in
[7:41:03] most of the real case scenario, you are going to have a pre-prepared data. It may be structured, semi-structured, or unstructured. But for this tutorial, we are going to create a random data for the product
[7:41:17] based on the given region and also the date and time. Also, if you want to try the simulation to generate a random data based on the given dates, regions, and the products. So, you're going to get a code something
[7:41:30] like this. Now, let us dive deep into this given tutorial and try to understand what are the libraries used in this given code. You can see it has used Pandas, NumPy, and Matplotlib. And also, guys, you can
[7:41:44] see it is trying to create the data with the help of NumPy. So, you can see there are dates, regions, and products. Then there is a for loop, and it is rows. So, you can see with date, region,
[7:41:58] products, unit sold, price, and revenue. And finally, you are storing this in a data frame. Now, let us try to run this code.
[7:42:10] Now, let us move to the next step and try to do some analytics. Now, guys, let us give a prompt to chat GPT. Something like this, saying that I have pandas
[7:42:22] data frame named sales with column date and region, and also with products, unit sold, price, and revenue. Try to give a brief uh idea about what data you have to chat GPT. And then say write simple pandas code to answer what is the
[7:42:37] average revenue per region, or return a small table with column region and lowest. Let us try to hit the button, and let's see. Now, you can see all over here, we have got a code. So, we can just copy
[7:42:52] this and try to paste on our collab. So, now let us try to run this, and you can see we have got the given output with the region and the average revenue, like west, south, north, east, and all those
[7:43:05] things. Let us move to the next analytics part. But before that, I would like to ask you some questions. Now, here's a little part I want you to try. You can also ask all over here that show
[7:43:20] the total revenue by product, and which day had the highest revenue overall. you. So, it it's mostly it's becoming a tool-based, and uh but with the complexity of the data,
[7:43:35] things can become little bit complex. But since we have the GenAI, now your work will become very much easier. So, do try this, guys, and mention in the comment section below how many of you have got the right result. Now, as
[7:43:51] we all know that real data is actually very messy. So, guys, I have added some impurities in the given data. You can see I have made some missing values, I have made some duplicate rows, and I have also some lowercase and spaces.
[7:44:06] So, that makes a pretty interesting how GenAI is going to deal with this Let us try to hit the run button and first try to insert all those thing, and you can see the data has been added. Now, let us try to do some analysis.
[7:44:22] Now, let's say in a given scenarios, first you need to understand your data. Where are the missing values and what are the issues in your given data? And then you can give prompt something like this. Say, I have pandas data frame
[7:44:36] messy like before, please write a simple readable code and trim spaces and fix regions like title case, drop the duplicate rows, and fill the missing revenue with unit sold. And make sure your date is in proper
[7:44:49] data type, and then return the clean data frame named clean. Now, let us give another prompt to generate a chart with the help of GenAI. Basically, I'm using and the model is five. So, you can write a prompt something
[7:45:05] like this. Write a simple matplotlib code to plot a bar chart of the top five products by total revenue from the data frame clean, and also add the title and label the axis. So, clean is our data frame.
[7:45:20] Let's click enter. You can see GPT has easily given us the code. Just copy go to the terminal and just paste the given code, given code, and just click on enter.
[7:45:36] have got a very amazing visualization. You can see there is total revenue, revenue, and you can see this is watch, laptop, phone, headphones. So, in this simulation, you can see how easily you can generate charts.
[7:45:55] very much interesting. So, let us say I asked the GPT something We'll keep the forecasting simple. We will use total revenue by month, then use a 3-month average to guess the next
[7:46:09] one. So, you can type a prompt something like this, that we want forecasting, we will use total revenue by month, then use a 3-month average to guess the next one. Just click enter after that. And you can see it is going to give you
[7:46:24] the code, which is going to generate the forecasting. step, calculate the total revenue by month, then calculating the 3-month moving average, and finally, it has done the forecasting of the next month.
[7:46:39] Now, let us copy this given code and paste on the Google Colab. So, we have copied our given code. Let's try to run it.
[7:46:53] have got the output, and it has generated the forecasting of our given product. So, you can see from this graph, guys, this is the revenue Y axis, and on the X axis, we have the actual revenue, and the dash lines are 3-month
[7:47:06] average. Now, you can see from all over here, we have the given years, like 2024, first month, the third month, fifth month, seventh month, ninth month, And it is creating the monthly revenue
[7:47:21] versus 3-month So, that is amazing analysis, and we have just got to do it in few seconds by writing the prompt. Now, you would be also thinking about the same thing which I am thinking, like
[7:47:36] GenAI is definitely going to make a lot of things easier in the domain of data analytics. Hello, everyone, I am Mayank and welcome to today's video where we will be talking about LLM benchmarks, tools used to test and measure how well
[7:47:50] large language models like GPT and Google Gemini performs. If you have ever wondered how AI models are evaluated, this video will explain it in simple terms. LLM benchmarks are used to check how good these models are at tasks like
[7:48:04] coding, answering questions, and translating languages, or summarizing text. These tests use sample data and a specific measurement to see how well the model perform. For example, the model might be tested with a few examples,
[7:48:16] like few-shot learning, or none at all, like zero-shot learning, to see how it handles new tasks. So now the question arises, why are these benchmarks important? They help developers understand where a model is strong and
[7:48:29] where it needs improvement. They also make it easier to compare different models, helping people choose the best one for their needs. However, LLM benchmarks do have some limits. They don't always predict how well a model
[7:48:41] will work in real-world situation, and sometimes models can overfit, meaning they perform well on test data, but they struggle in practical use. We will also different models based on their benchmark scores, giving us a clear
[7:48:55] picture of which models are performing the best. So stay tuned as we dive into how LLM benchmarks work and why they are so important for advancing AI. So without any further ado, let's get started. So what are LLM benchmarks? LLM
[7:49:07] benchmarks are standardized tools used to evaluate the performance of large structured way to test LLMs on a specific task or question, using sample data and predefined metrics to measure their capabilities. These benchmarks
[7:49:20] assess various skills such as coding, common sense reasoning, and NLP tasks answering, and text summarization. The importance of LLM benchmarks lies in their role in advancing model development. They track the progress of
[7:49:34] an LLM, offering quantitative insights into where the model performs well and feedback is crucial for guiding the fine-tuning process, allowing researchers and developers to enhance model performance. Additionally,
[7:49:48] benchmarks offers an objective comparison between different LLMs, helping developers and organizations choose the best model for their needs. So, how LLM benchmarks work? LLM benchmarks follow a clear and systematic
[7:50:01] process. They present a task for LLM to complete, evaluate its performance using specific metrics, and assign a score based on how well the model performs. So, here is a breakdown of how this process work. The first one is setup.
[7:50:15] LLM benchmarks come with pre-prepared sample data, including coding challenges, long documents, math problem, and real-world conversation. The task expand various areas like common sense reasoning, problem-solving,
[7:50:28] question-answering, summary generation, and translation, all presented to the model at the start of testing. The second is step is testing. The model is tested on one of the three ways. Few-shot, the LLM is provided with a few
[7:50:40] examples before being prompted to complete a task, demonstrating its ability to learn from limited data. The second one is zero-shot. The model is asked to perform a task without any prior examples, testing its ability to
[7:50:52] understand new concept and adapt to unfamiliar scenarios. The third one is fine-tuned. The model is trained on a data set similar to the one used in the benchmark, aiming to enhance its performance on the specific task and
[7:51:05] work. The third step is the scoring. So, after completing the task, the benchmark expected answer and generates a score, typically ranging from 0 to 100, reflecting how accurately the LLM performed. So, now let's moving forward,
[7:51:19] let's see key metrics for benchmarking LLMs. So, LLM benchmark uses various metrics to assess performance of large language model. So, here are some commonly used metrics. The first one is accuracy or precision. Measure the
[7:51:32] the model. The second one is recall, also known as sensitivity, measure the number of true positive reflecting the correct prediction made by the model. The third one is F1 score, combines both accuracy
[7:51:46] and recall into a single metric, weighing them equally to address any false positive or negative. F1 score ranging from zero to one, where one indicates perfect precision and recall. The fourth one is exact match. Tracks
[7:51:58] the percentage of predictions that exactly match the correct answer, which is especially used for the task like translation and question answering. The fifth one is perplexity gauges. Here, it will tell you how well a model
[7:52:11] predicts the next word or token. A lower perplexity score indicates better task comprehension by the model. The sixth one is BLEU, Bilingual Evaluation machine translation by comparing n-grams, sequence of adjacent text
[7:52:26] element, between the model's output and the human produced translation. So, these quantitative metrics are often combined for more thorough evaluation. So, in addition, human evaluation introduces qualitatively factors like
[7:52:38] coherence, relevance, and semantic meaning to provide a nuanced assessment. However, human evaluation can be time-consuming and subjective, making a balance between quantitative and qualitative measures important for
[7:52:50] comprehensive evaluation. So, now let's moving forward see some limitation of are valuable for assessing model performance, they have several fully predicting real-world effectiveness. So, here are some few.
[7:53:05] The first one is bounded scoring. Once a model achieves the highest possible scores on the benchmark, the benchmark loses its utility and must be updated with more challenging tasks to remain a meaningful assessment tool. The second
[7:53:17] one is broad data set. LLM benchmark often rely on sample data from diverse subject and task. So, this wide scope may not effectively evaluate a model performance in such cases, specialized fields, or specific use cases where more
[7:53:30] tailored data would be needed. The third one is fine-grained assessment. Benchmark only test a model's current skills, and as LLMs evolve and the new capabilities emerge, new benchmarks must be created to measure these
[7:53:42] advancements. The fourth one is overfitting. So, if an LLM is trained on can be lead to overfitting, where the model performs well on the test data but it struggles with the real task. So, this result is scores that don't
[7:53:56] truly represent the model's broader capabilities. So, now what are LLM leaderboards? So, LLM leaderboards publish a ranking of LLMs based on a provide a way to keep track to the myriad LLMs and to compare their
[7:54:10] performance. LLM leaderboards are especially beneficial in making decisions on which models to use. So, here are some. So, in this you can see here are some. So, in this you can see here OpenAI is leading and GPT-4o second
[7:54:22] and the Llama third with 405 parameter B and 3.5 Sonnet is there. So, this is best in multitask reasoning. What about the best in reasoning. What about the best in coding? So, here OpenAI 01 is leading. I
[7:54:35] guess this is the Orion 1, and the second one is 3.5 Sonnet, and after that in the third position there is GPT-4o. So, this is in best in coding. The next comes fastest and most affordable models.
[7:54:49] So, fastest models are Llama 8B parameter, 8B parameter. And the second parameter, 8B parameter. And the second one is LLaMA Llama 70B. And the third one is 1.5 Flash. This is Gemini 1. And lowest latency.
[7:55:04] And here, it is leading Llama again. In cheapest models, again Llama 8B is leading. And in the second number, we have Gemini Flash 1.5. And in third, we have GPT-4o mini.
[7:55:17] Moving forward, let's see standard benchmarks between Claude 3 Opus and GPT-4. So, in general, they are equal. In reasoning, Claude 3 Opus is leading, and in coding, GPT-4 is leading. In math, again, GPT-4 is leading. In tool
[7:55:33] use, Claude 3 Opus is leading, and in multilingual, Claude 3 Opus is leading. >> So, let's talk about the very first top AI tool, which is Skillora.ai. This AI tool aims to provide the best AI-driven
[7:55:47] mock interview experience. You can take mock interviews for any job roles and subject, receiving instant constructive feedback. Skillora's AI interviewer listens to your responses and asks dynamic flow of questions to test your
[7:56:01] depth of knowledge. With the essential plan, you gain access to six AI mock interviews per month. You'll receive instant feedback, ideal answers, and performance reports, helping you upgrade your skills for just
[7:56:14] $8 per month. The premium plan allows you to enjoy 15 AI mock interviews per month. You'll get the same instant feedback, ideal answers, and performance reports, along with priority support, maximizing your learning for only $15
[7:56:28] per month. Next up, we have Viscose. Viscose's advanced AI coach is designed to elevate your job interview skills, harnessing insights from over 10,000 It offers tailored sessions that simulate real interviews, providing
[7:56:42] personalized feedback and performance analytics. It transcribes your performance and delivers a conversational experience akin to an actual interview. It also offers customized feedback and performance
[7:56:54] analytics, helping you refine both your answers and communication skills. During practice session, you can craft questions specific to your desired role and industry, ensuring relevance and depth in your preparation, supporting
[7:57:07] various interview styles. The price for Eva's services are available for free. Next up, we have interviewing.io. Interviewing.io is an AI-powered platform that provides candidates with
[7:57:19] simulated technical interviews. The platform uses machine learning algorithms to access your coding skills, problem-solving abilities, and communication style. You can get four simulated technical interviews for $225.
[7:57:33] Next up, we have Mockmate. Mockmate is an AI powered tool that allows you to practice answering common interview questions. This tool provides feedback on your answers and gives tips on how to improve your performance. The service is
[7:57:46] available for $29 per month. Moving on to ChatGPT. ChatGPT is an AI chatbot that can be used to practice answering interview questions in a conversational setting. The chatbot can be customized to
[7:57:59] interview, and it provides feedback on your answers. The free plan allows you to get started with writing assistant, problem-solving, and many more. You will have limited access to ChatGPT 4o mini, including data analysis, file uploads,
[7:58:14] vision, web browsing, and custom GPTs. All of this is available at no cost. Then we have the plus plan, which is for $20 per month. The plus plan amplifies your productivity by offering early access to new features, access to
[7:58:29] access to new features, access to ChatGPT 4 4o and GPT 4o's mini, up to 5x more messages for ChatGPT 4o. You can access to data analysis, upload files, vision, web browsing, and DALL-E image generation. You can also create and use
[7:58:42] custom GPTs. Then we have My Interview Practice. My Interview Practice is an AI tool that allows you to practice interviews in a simulated environment. The platform uses AI to generate interview questions based
[7:58:55] on your chosen field, and provides instant feedback on your responses. This tool is particularly useful for those who want to practice their interview skills in low-pressure environment. The plus plan is for $49 per month. You get
[7:59:09] 1 month of access. And the premium plan is for $57 per month and you will get more benefits. Next up, we have Yoodli. Yoodli offers
[7:59:21] real-time interview coaching using AI. It analyzes your speech and body language, providing feedback on how to improve your performance. Yoodli's services are available for free. Now, you might be wondering how AI tools
[7:59:34] can help you ace your job interviews. AI tools can assist in several ways. At first, we have research companies. AI tool provides information about companies' website, news articles, and social media, helping you tailor your
[7:59:48] answers to the interview's question. Practice questions. AI tool help you practice answering common interview questions, boosting your confidence and preparation. Feedback. AI tool offers feedback on
[8:00:00] improvement, and refining your responses. Personalized advice. AI tools provide tips and advice on improving your interview skills, giving you an edge to your next job interview. Choose the
[8:00:13] right AI tool and select an AI tool that fits your needs and preferences. Use the tool consistently, starting a few weeks before your interview to maximize its benefits. After practicing with the AI tool, get feedback from others to
[8:00:27] identify areas for further improvement. Dress professionally and be punctual for your interview to make a positive first impression. Show enthusiasm and confidence during the interview to leave a lasting impression on the interviewer.
[8:00:41] AI tools can significantly aid in interviewing preparation. By using them effectively, you can enhance your chances of success in your next job interview. Hey everyone, have you noticed how the word AI is being thrown
[8:00:53] around almost everywhere today? From marketing posts to casual conversations, people are saying things like, "We use AI. We built an AI tool." or "AI is the future." But, here's the truth. Most of these people have no real understanding
[8:01:09] of what AI actually is and that's the problem because AI is not a magic. It's not just a trend. It's built on real foundations, mathematics, logic, and code. If we are serious about working in AI or building with AI, we need to
[8:01:25] understand the core of how it actually works and that's exactly why we are making this video. The goal here is very simple, to bring clarity. So, in this video, we will give you 15 basic questions about AI. Not advanced stuff,
[8:01:41] not industry buzzwords, just the real absolute fundamentals that every beginner or aspiring AI professionals must know. No fluff, no jargons, and no shortcuts. These questions are meant to check if you truly understand what AI
[8:01:57] actually is, how it works, and what's going on under the hood. So, if you don't know the answer to some of these, that's okay. It just means you now know exactly what to go back and learn because the truth is if someone can't
[8:02:12] answer these 15 basic questions, they are not ready to work in AI yet and that's not gatekeeping. That's just being honest about what this field requires. Now, before we dive into the core questions, take a moment to really
[8:02:26] think about each one. Don't rush through them. Make sure you subscribe to our channel and drop a like and share if you want us to make more such videos. Let's get into it. Let's have a look at our first question, which is what does it
[8:02:40] actually mean for a machine to learn? Now, here's a quick explanation of this question. We say machines learn from data, but what does learning really mean data, but what does learning really mean in this context?
[8:03:01] understand anything or it is just advanced pattern recognition? Let's have an explanation to this question. Now, this question challenges whether AI has any kind of awareness or if it's just calculating probabilities.
[8:03:24] would happen if we trained an AI model on completely incorrect or biased data? Now, this question isn't just theoretical. This happens in the real theoretical. This happens in the real world. Think about the consequences.
[8:03:46] build an AI that makes decisions without being explicitly told how? Now, I'll give an explanation of this question. So, this goes to the heart of what intelligence mean in machines and whether rules are always required.
[8:04:08] do you know if an AI system is being smart or just memorizing? Let's have an explanation of this question. Now, this is a real challenge in evaluating AI performance. Is the system learning concepts or just recalling patterns? You
[8:04:24] concepts or just recalling patterns? You need to know about this. question, which is, what is overfitting and why is it a problem? Now, you can
[8:04:38] think of it as a model that memorizes instead of learning can feel badly in instead of learning can feel badly in real world.
[8:04:51] is, what does an activation function do in a neural network? Let's have an explanation to this question. It's a small component with a huge impact on small component with a huge impact on how neural networks work.
[8:05:11] learn from completely random data? Now, this question can be tricky because it makes you think about how learning really works.
[8:05:27] descent and why do we use it? Now, you can think of this question about if you have ever trained a model, you have probably used this, but do you probably used this, but do you understand it or you not?
[8:05:46] and the quality of the data set matter so much? Now, the explanation for this question is, you can't make something smart with garbage input, and yet many smart with garbage input, and yet many try.
[8:06:04] difference between a model and an algorithm? Now, you can think of it like this. These two get confused a lot, but they are not the same thing.
[8:06:22] model performs well on training data but fails on real-world data? Now, for this question, this scenario is more common than you would think, and it would be than you would think, and it would be way more dangerous.
[8:06:41] function? Let's have an explanation to this question. It's not just some technical detail, it literally tells the model how wrong it is.
[8:06:58] Question number 14. Can a machine learning model be 100% accurate? Now, this is one of those questions that seems simple, but the implications are seems simple, but the implications are deep.
[8:07:16] in AI and where do they come from? Let's have an explanation to this question. Now, this question goes beyond math and into real-world consequences.
[8:07:34] of our course. If you have any doubts or questions, ask them in the comment section below and our team of experts will be happy to help you as soon as possible. If you found this course helpful, do like this video and don't
[8:07:47] forget to subscribe for more such courses. Until next time, thank you for watching and keep learning with Simply Learn.