---
title: 'Agentic AI Full Course in 8 Hours 2027 | Complete Agentic AI Course For Beginners | Simplilearn'
source: 'https://youtube.com/watch?v=Pz1DTbSeqwY'
video_id: 'Pz1DTbSeqwY'
date: 2026-11-07
duration_sec: 29282
channel: 'Simplilearn'
---

# Agentic AI Full Course in 8 Hours 2027 | Complete Agentic AI Course For Beginners | Simplilearn

> Source: [Agentic AI Full Course in 8 Hours 2027 | Complete Agentic AI Course For Beginners | Simplilearn](https://youtube.com/watch?v=Pz1DTbSeqwY)

## Summary

The video is a comprehensive but fragmented course on agentic AI, covering fundamental concepts like generative AI and large language models, as well as more advanced topics like AI agents, workflows, and tools. However, it is presented as a series of disjointed tutorial introductions rather than a cohesive, in-depth educational experience, with significant portions devoted to promotional content.

### Key Points

- **Introduction to Agentic AI** [00:08] — Agentic AI is defined as an AI system that can understand goals, plan tasks, and complete complex workflows with minimal human intervention. It is changing how we interact with technology by automating processes and building intelligent assistants.
- **What is Generative AI?** [04:33] — Generative AI refers to AI designed to create new content, such as text, images, and music, based on patterns learned from vast data sets. Tools like ChatGPT and DALL-E are examples, and its applications range from content creation to healthcare simulations.
- **Large Language Models Explained** [10:06] — Large language models (LLMs) are sophisticated AI systems trained on vast datasets using deep learning and transformer architectures to understand and generate human-like text. Their applications include powering virtual assistants, chatbots, content creation, and language translation.
- **Understanding AI Agents** [19:58] — AI agents are advanced virtual assistants designed to perform tasks autonomously. They automate repetitive tasks, enhance decision-making with data-driven insights, and act as personalized digital assistants across various tools and apps.
- **OpenAI's New Agents** [33:19] — OpenAI released two groundbreaking agents: Operator, which mimics human actions on a browser using a new CUA model, and Deep Research, which performs comprehensive research and compiles information into detailed reports using the O3 model.
- **The Four Steps of Agentic AI** [44:42] — Agentic AI works in four steps: perceiving (gathering data from databases, sensors, APIs), reasoning (using LLMs to interpret tasks and generate solutions), acting (executing tasks autonomously via external tools), and learning (improving through feedback loops or data flywheels).
- **Generative AI vs. Agentic AI** [35:37] — Agentic AI differs from generative AI by not just generating responses but also executing tasks autonomously. It is adaptive, learning from its actions, and requires minimal human involvement, unlike generative AI which relies on user prompts.
- **What is MCP?** [03:50] — MCP (Model Context Protocol) is an open-source protocol that standardizes how applications supply context to large language models, simplifying the integration of AI models with various data sources and tools.

## Transcript

Imagine an AI system that could understand your goals, plan tasks, make complete complex workflows with minimal human intervention. Sounds interesting, right? Well, that's exactly what agentic AI is
all about. From automating repetitive business processes to building intelligent assistants, analyzing data, and creating AI-powered applications, agentic AI is changing how we interact with technology. Businesses are
exploring ways to use AI agents to improve productivity, automate workflows, and solve problems that traditionally require multiple manual steps. But here's the big question. How do these AI agents actually work, and
how can you build one yourself? Hey everyone, welcome to this agentic AI full course by Simply Learn. In this comprehensive course, we'll take you through the fascinating world of AI agents, starting with the fundamentals
applications. You'll begin by understanding what AI agents are, how they interact with different environments, and how agentic AI workflows operate. We'll also explore essential concepts like natural language
processing, generative AI, reinforcement learning, and Q-learning. As we move forward, you'll learn how to build AI agents from scratch, create AI systems that combine information retrieval with intelligent AI workflows.
platforms like LangChain, LangGraph, LangFlow, LangSmith, Hugging Face, and roles in developing AI-powered solutions. You'll also explore AI tools and models, including Manifold AI, Deep Seek, Llama, Claude, and Gemini, along
with practical topics like Python-based AI web scraping, AI video generation, and generative AI for data analysis. We'll discuss career opportunities, the generative AI engineer roadmap, and ways AI agents can create business value.
Whether you're a beginner exploring AI or a developer looking to expand your skills, this course will help you understand the concepts, tools, and workflows behind agentic AI. That said, if these are the types of videos you
subscribe button along with the bell icon to get notified whenever we upload a new video. Before we start, if you're looking to build deeper expertise in agentic AI and learn how autonomous AI systems are designed for real-world use,
here's something worth checking out. The Microsoft Applied Agentic AI Systems Design and Impact Program is designed for professionals who want to move beyond basic generative AI and learn how to build and manage
production-ready AI agents and multi-agent systems. Through live practitioner-led sessions, hands-on projects, guided practices, and a capstone project, you'll build practical experience with agent design,
orchestration, rag, MCP, planning systems, workflow automation, and The program also includes a joint completion certificate from Microsoft and Simply Learn, along with Microsoft Learn credentials and Microsoft branded
agentic AI learning content. Practical use cases involving agentic rag systems, multi-agent AI-powered automation, agentic UX, financial analysis, and production-grade go-to-market systems helping you understand how autonomous AI
solutions are actually designed and deployed. So, if you want to build production-ready agentic AI skills and gain practical experience creating autonomous AI systems for real business scenarios,
check out the link in the description box below and in the pinned comments. Now, let's get started. What is the primary purpose of an AI agent? A, to store large amounts of data without processing it.
B, to perceive its environment, make decisions, and take actions toward achieving specific goals. C, to perform only mathematical calculations without interacting with external systems. D, to replace all programming languages with
natural language. Let us know your answer in the comment section below. Let's get started with this course. &gt;&gt; Meet Emma, a graphic designer working on a new project. One day, her colleague mentions a tool that helps create
mentions a tool that helps create designs, images, and text using AI. Intrigued, Emma wonders how AI can create something from scratch. Her curiosity grows, and she decides to dive deeper into this new technology
called generative AI. Generative AI refers to a type of artificial intelligence designed to create new content, such as text, images, music, and videos. Unlike traditional AI, which analyzes or
categorizes data, generative AI produces original content based on patterns learned from vast data sets. Essentially, it generates new, unique material. These models are often trained on large
amounts of data and use sophisticated algorithms to mimic human creativity. Tools like ChatGPT or DALL-E can create art, write essays, or simulate conversations by generating outputs based on user prompts.
Generative AI has a wide range of applications. Content creation. Tools like GPT-4 generate text, blog posts, stories, and essays from simple Art and design. AI models such as DALL-E generates
unique images and designs based on text descriptions, transforming creativity in Music and audio. AI can compose music or replicate voices, offering new possibilities for musicians and audio engineers.
Generative AI simulates disease progression or create synthetic medical data, helping doctors gain faster insights for research. Let's take image generation as an example to explain how generative AI
works. Data collection and learning. AI models like DALL-E are trained on large data sets of images paired with text descriptions. These data sets teach the model to recognize different
objects, colors, styles, and how to associate text with corresponding The more data the AI learns from, the better it can generate accurate and better it can generate accurate and diverse images based on user prompts.
Neural networks and transformers. When Emma inputs a prompt like a cat wearing sunglasses, the transformer model processes the text, recognizing words like cat and sunglasses, and links them to images it learned from during
training. Transformers help the AI decide how to image. Tokens and context. The text input, such as a cat wearing sunglasses, is split into smaller parts
called tokens. The AI processes each token and understands the relationship. For instance, it knows the sunglasses should be placed on the cat, creating a contextually accurate image. Feedback mechanism.
Generative AI models improve through feedback. After generating an image, users provide feedback on the accuracy or quality of the output. If Emma's generated image shows the sunglasses floating beside the cat, she can mark it
The model uses this feedback to improve future image generations. Reinforcement learning further enhances The model's rewarded when it generates accurate images and corrected when it
For example, when Emma describes a sunset and the AI produces a vibrant sunset image, it receives positive reinforcement. model's ability to generate better images.
Data science and AI models. Data scientists curate the training data and define the parameters that help the AI generate accurate images. versatile the AI becomes in generating diverse types of content.
parameters, which are settings that guide the AI in processing data and generating outputs. Generating original content. Once trained, the model can generate original images. For example, you then
might describe a futuristic cityscape, and the AI would produce a unique image based on what it learned. The generated image isn't just a copy of past data, but an entirely new creation, showcasing the AI's ability to combine
learned patterns and creativity. &gt;&gt; Large language models, LLMs. If you ever wondered how machine learning can now understand and generate human-like text, chatbots like ChatGPT to AI assistant that power search engines, LLMs are
transforming how we interact with technology. One of the most exciting advancement in this space is Google's Gemini or OpenAI's ChatGPT, a designed to push the boundaries of what AI can achieve. In this video, we will
explore what LLMs are, how they work, and why models like Gemini are critical for the future of AI. Google Gemini is part of a new wave of AI models that are smarter, faster, and more efficient. It is designed to understand context
better, offer more accurate responses, and integrate deeply into service like Google Search and Google Assistant, providing more human-like interactions. behind LLMs, including their massive training data set, transformer
architecture, and how models like Gemini use deep learning innovation to change industries. Plus, we will compare Google Gemini to other popular LLMs, such as open AI chat GPT models showing how each of these technologies is used to power
chatbots, virtual assistants, and other AI driven applications. By end of this understanding of how large language models like Gemini work, their key features, and what they mean for the
subscribe, and hit the bell icon to never miss any update from Simply Learn. So, without any further ado, let's get started. So, what are the large language models? Large language models like chat GPT 4 generate a pre-trained transformer
4 0 and Google Gemini are sophisticated AI systems designed to comprehend and are built using deep learning techniques and are trained on vast data set leverage self-attention mechanism to analyze relationship between words or
tokens allowing them to capture context and produce coherent relevant responses. LLMs have significant application including powering virtual assistant, chatbots, content creation, language translation, and supporting research and
decision making. Their ability to generate fluent and contextually language processing and improved human-computer interaction. So, now let's see what are large language model used for. Large language models are
utilized in scenarios with limited or no domain specific data available for training. These scenarios include both few-shot and zero-shot training approaches which rely on the model's strong inductive bias and its capability
to derive meaningful representation from a small amount of data or even no data at all. So, now let's see how are large language model trained. Large language models typically undergo pre-training on a boat all encompassing data set that
shares statistical similarities with the data set is specific to the target task. The objective of pre-training is to enable the model to acquire high-level feature that can later be applied during the fine-tuning phase for specific task.
So, there are some training processes of LLM which involves several steps. The first one is text pre-processing. The textual data is transformed into a model can effectively process. This conversion may be involved techniques
like tokenization, encoding, and creating input sequences. The second one model's parameter are initialized randomly before the training process begins. The third one is input numerical data. The numerical representation of
processing. The model's architecture, typically based on transformers, allows it to capture the contextual relationship between the words or tokens function calculation. A loss function calculation measures the discrepancy
between the model's prediction and the actual next word or token in a sentence. The LLM model aims to minimize this loss during training. The fifth one is parameter are adjusted through optimization technique. This involves
parameters accordingly, gradually improving the model's performance. The last one is iterative training. The training process is repeated over multiple iterations or epochs until the model's output achieve a satisfactory
level of accuracy on the given task or data set. By following this training process, large language models learn to capture linguistic patterns, understand responses, enabling them to excel at various language-related tasks. The next
topic is how do large language models work? So, large language models leverage deep neural network to generate output based on patterns learned from the language model adopts a transformer architecture, which enables the model to
identify relationship between words in a sentence irrespective of their position in the sequence. In contrast to RNNs that rely on recurrence to capture token relationship, transformer neural network employs self-attention as their primary
attention scores that determine the importance of each token with respect to the other token in the text sequence, facilitating the modeling of intricate relationship within the data. Next, let's see application of large language
wide range of application across various domains. So, here are some notable language processing, NLP. Large language language understanding tasks such as sentiment analysis, named entity
recognition, text classification, and language modeling. The second one is chatbot and virtual assistant. Large language models power conversational agents, chatbots, and virtual assistant, providing more interactive and
human-like user interaction. The third one is machine translation. Large language models have been used for automatic language translation, enabling text translation between different languages with improved accuracy. The
fourth one is sentiment analysis. LLMs can analyze and classify the sentiment or emotion expressed in a piece of text, which is valuable for market research, analysis. The fifth one is content recommendation. These models can be
employed to provide personalized content recommendations, enhancing user experience and engagement on platforms such as news website or streaming highlight the potential impact of large language models in various domains for
automation, and interaction between humans and computers. &gt;&gt; Now, the agenda for today's session. Firstly, let's get started with the agenda for today's session. Now, we will have a brief introduction
to this session. Followed by that, we will have a briefing of what exactly is AI, then how does it work. Next, we will understand the key technologies and frameworks involved, and we will also go for some everyday use cases where we can
take help of AI, and also some of the responsibilities that we need to keep in mind for the ethical use of AI, and lastly, we will explore some prompts. the agenda, let's get started with the first part, introduction.
How can AI simplify daily tasks? As we discussed in the use case, a human might have a lot of routine and mundane tasks. For example, automation. AI can simplify your daily tasks like checking up emails and drafting email responses, enhancing
productivity by getting AI assistance for research and marketing trends or any other researches based on your job profile. And you can have a personalized AI assistance for recommendations for learning and much more. And after that,
it will save you a lot of time reducing effort. Now, Get set up with generative AI models like Microsoft Copilot and try some of the examples. So, if you get access to one of the generative AI models like
GitHub, then you can access GitHub Copilot easily, use it in Microsoft 365 apps like Word, Excel, and Teams. Then you can also explore prompting basics to start up with simple commands like drafting emails or summarizing
documents. And proceeding ahead, we have another opportunity to experiment with different use cases like trying Copilot with brainstorming, planning, and organization. And fourth one, customize and iterate. Improvise AI-generated
results by refining prompts based on your needs and requirements. Now, let's proceed ahead and understand what exactly is AI and how it works, which will involve some of the key topics like understanding the basics and
technologies involved, etc. So, generative AI is an advanced type of artificial intelligence that can sync up with human activities and gather requirements to assist humans in their daily tasks, improving efficiency and
effectiveness. Not just these professional tasks, AI can create new content like text, images, music, and videos. It uses natural language processing and neural networks to learn patterns from existing data and then
patterns from existing data and then generate new content. Now, let's see how we can use AI to simplify our daily tasks and make life easier. Firstly, AI can create unique content, not just analyze data and make
predictions. It can assist you in real-time generation of original text, images, music, and more. Next is AI as a creative partner. Generative AI can and much more. It goes beyond traditional AI. Unlike
predictive AI, generative AI creates new content rather than just analyzing data. functionality where it can enhance human creativity by assisting with brainstorming ideas, storytelling, design, and content generation. Lastly,
it is widely used in multiple fields, found in marketing, education, entertainment, and software development. Now, let's proceed ahead and understand the key technologies behind generative AI. Large language models is the first
one. Trained on vast amounts of text to generate human-like responses. For example, ChatGPT, PaLM, LaMDA, and Microsoft Copilot. It is used for writing, summarization of code, and emails, and much more. Now, the next
emails, and much more. Now, the next part is natural language processing. It enables AI to understand, interpret, and generate human-type languages. It powers chatbots, voice assistants, and translation tools. Next, it helps
improve communication between humans and AI. Now, let's proceed ahead and understand the everyday use cases of AI assistants. Starting with creative writing and content creation. You can create and
generate blog posts, articles, social media captions, and scripts. AI assistant in brainstorming ideas, editing text, and improving clarity. And lastly, you can also create poetry, stories, or some song lyrics using AI.
The second one is organizing and planning. You can make use of AI to plan a trip itinerary based on preferences and budget. You can generate meal plans and grocery lists as well. It can schedule events and set reminders and
manage to-do lists. The next use case is personal assistance. You can use AI to draft and refine emails or messages. It also helps you by providing phone books and recommending movies and music. And
lastly, summarize documents, articles, or even meeting notes. Lastly, we have some other practical uses which include the following. Learning new skills with AI-generated study plans, translating languages to
provide real-time interpretations, offering health and fitness suggestions Now, proceeding ahead, we will understand AI agents. AI agents are advanced virtual assistants designed to perform tasks
autonomously. They interact with users, process information, and execute commands. Some of the examples are ChatGPT, Microsoft Copilot, Gemini, and others. Now, what do AI agents actually do?
So, AI agents conduct the following tasks. First, automate some repetitive tasks. Repetitive tasks like scheduling meetings, setting up reminders, and managing emails is done by AI assistants. It can enhance
decision-making by providing data-driven insights and recommendations. Third is it can act as a personalized digital assistant which can help you with learning, travel planning, productivity, and much more. Lastly, integrating with
tools and apps which can help you work across Microsoft 365, CRM platforms, and And now, let's proceed with the major part of today's session where we have to part of today's session where we have to focus on how to use AI responsibly and
effectively. So, here we have a detailed Microsoft Copilot AI overview and also productivity. Let's go through that first. So, here we have a quick briefing of Microsoft 365 Copilot and we have also
some links to the Copilot success kit, Copilot chat, agent starter kit, and the community links over here and resources, Copilot mobile version, what's new, and all and everything you need to know about Copilot, about the technical
Now, back to the presentation where we were discussing about the responsible use of AI. So, the importance of ethical use of AI. AI should be used in a way that it is fair, transparent, and accountable.
Understanding AI limitations will help you prevent misinformation and misuses. Ethical AI usage ensures trust and reliability in everyday applications. ensuring transparency and accountability. First one, clearly
communicate when AI is used in content creation. Provide with the specific prompts that you're expecting in your output so that it cannot miss on the Secondly, we need to verify AI-generated information before sharing or acting on
it because there might be instances where we might go wrong explaining our inputs or there might be some discrepancy in the data provided to analyze and generate a report. And lastly, understand how AI makes
decisions to avoid over-reliance. Need to understand the thought process of an AI and expect what kind of results it might generate so that you don't have to rely on one single prompt every single time. Now, let's move ahead and address
some potential biases in AI. AI can sometimes reflect biases present in its training data. Common biases include gender bias, racial bias, cultural bias, confirmation bias, and socioeconomic biases. Let's discuss each
one of them. Firstly, gender bias. AI-generated job descriptions may reinforce stereotypes, example, associating engineers' roles with men. Second one is racial bias. Facial recognition systems
may misidentify individuals from certain ethnic backgrounds. Third one, cultural bias. AI might prioritize Western perspectives in content generation. Fourth one, confirmation bias. It may reinforce existing beliefs by favoring
information similar to past searches. Lastly, socioeconomic bias. For example, AI-generated loan approvals may favor individuals from higher-income Now, you also need to focus on responsible prompting. Use clear and
neutral language to avoid influencing biased results. Ask AI to provide diverse perspectives and sources when generating responses. Be mindful of data privacy and avoid sharing sensitive personal information. Now, let's go
ahead and check some of the prompt examples. So now, we are on the Microsoft Copilot prompt gallery, which will show you some of the sample prompts that you can make use of. For example, get a to-do list,
add an image, quiz yourself, create a technology, write more confidently. For example, you might have a description, and you can just add it to improvise or add more impact onto it, find a specific information, draft an email, create a
shopping list, and so on. So here, you can also go through some other prompts as well by clicking on view all prompts and a whole new world of prompt gallery will open up for you. Try it out and let us know your experience in the comment
&gt;&gt; Are you feeling like every other video, article, or tweet you see these days is about generative AI? You're not wrong. It's the tech world's current obsession. But instead of just nodding along, wondering what it all means, what if you
could actually master it? In this video, we're cutting through the hype, giving you a practical step you need to truly understand. I will be breaking down the entire journey into manageable phases, from fundamental knowledge to building
your own Gen AI projects. Before we begin, let me tell you what generative AI or Gen AI means. As the name suggests, it is generating data. It's a technology where it uses existing data for creating new ones. It might be in
the form of text, images, audio, video, code, or even 3D models. For better understanding, I will be dividing the entire road map into four phases, plus one additional phase as a bonus phase for you to excel in this field. Phase
one is all about laying the foundation, which includes the basics of AI and machine learning. Think of artificial intelligence as making computers smart enough to do things which people can do. Machine learning is just one of the way
to do it, allowing computers to learn from example instead of being told exactly what to do. It's a learning by seeing and doing it, not just by a rule book. There are three main types of machine learning: supervised learning,
unsupervised learning, and reinforcement learning. Supervised learning is where the computers learn with correct answers, whereas unsupervised learning involves finding patterns on its own, and reinforcement learning is about
learning through rewards. You will need Python too. Key tools include NumPy for numerical operations, Pandas for organizing data, and Matplotlib and Seaborn for data visualization. These help you to work with and understand
data. You need to understand data preparation, which involves cleaning and organizing data. Also, learn about the feature engineering, which is the process of selecting important parts of the data of the model. Gen AI uses deep
learning. You can think of deep learning as a computer network with many layers. Understand the basic idea of how these networks learn from data. This is a key for understanding more complex Gen AI models.
Moving on to phase two, where you can start exploring the Gen AI models. Some of Gen AI models includes GANs, which are two networks competing, one making fake data and other tries to spot it. This model works well for creating
realistic outputs. The second model is VAEs. This model learns a hidden representation of data to create new or similar data. The next model is transformers, which are excellent at understanding context and sequence like
text, audio, and even images using attention. The last model is diffusion model, which can create high-quality data by reversing a noising process. This model is very popular for images. Apart from Gen AI models, you need to
cover text generation model, which falls under natural language processing, which is NLP. These models power text creation from writing articles to translating languages. And lastly, you will cover image generation using GANs and
diffusion models. In phase three, we will focus on developing our practical skills. You can start with small projects such as generating basic images or text with small data set. Do experiment and learn the workflow. Join
platforms like Hugging Face, Reddit, Stack Overflow, and Kaggle forums to ask question and learn from others. To showcase your work, you can use GitHub to share your projects and code. Phase four is truly about staying ahead. Gen
AI is always changing, so you need to be aware of it. New architectures are constantly being built, so briefly be aware that more advanced models are continuously being developed beyond the basics that we have covered. I would
also recommend learning about deploying models on cloud platforms like AWS, Google Cloud, or Azure, which will be useful for larger projects. Additionally, you can read research papers, follow tech expert online,
whether on Twitter or LinkedIn, and stay curious about this field. Continuous learning is the key in this field. Now that you have built a strong foundation in generative AI, phase five is all about mastering the future of
technology. A certification course provides formal validation of specialized knowledge and skills, making individuals stand out to employees and demonstrating a commitment to professional development. This
recognition often leads to enhanced credibility that improves career opportunities with specific field. Wondering which certification to go with? Don't worry, I've got you covered here as well. That's your road map to
understanding generative AI. Start with the basics and get practical experience, and never stop learning. The power of Gen AI is within your reach. &gt;&gt; Hey everyone, welcome to Simply Learn's YouTube channel. In this tutorial, we
YouTube channel. In this tutorial, we will learn about OpenAI's GenAI agents. So, generative AI agents are advanced AI-driven systems designed to autonomously perform tasks, generate content, and assist in decision-making
by leveraging large language models, or also known as LLMs, and machine learning algorithms. Imagine an AI agent automating repetitive processes, enhancing productivity, and providing personalized recommendations across
various domains, including customer support, marketing, and software development. By understanding context and generating human-like responses, they improve user engagement and streamline workflows. Businesses benefit
from cost savings, faster problem solution, and improved efficiency. Such are a few out of many use cases of AI agents. Today, we will cover the OpenAI GenAI agents and their facts, their challenges, and future implications.
videos you'd like to watch, then hit that like and subscribe buttons and the bell icon to get notified. Open AI Gen AI agents. So, first, let's get started
will have a brief introduction to generative AI agents. Followed by that, we will understand the core components of Gen AI agents, then how generative AI agents work, architectures and deployment, enhancing AI agents with
external tools, challenges and limitations of Gen AI agents, future trends and advancements in AI agents will be the last one. So, I hope I made get started with the first part, which is introduction to generative AI agents.
So, basically, what exactly do you mean by the term generative AI agents? So, an AI agent is just an LLM that can take actions, and it can do so
autonomously without any human supervision, and now also reason about the tasks. An AI agent is software entity that autonomously interact with environments, make decisions, and execute tasks. Generative AI agents
execute tasks. Generative AI agents extend by leveraging large language models like Open AI's GPT to generate intelligent responses and perform tasks Now, we will also look into the evolution of
AI agents. Traditional rule-based AI agents had limited to predefined rules. Like, if you define a process and give the steps, it will just perform them and complete it. It will not take a step beyond it. And machine learning-based AI
agents can learn from data, but need a structured inputs. And then comes the generative AI agents, which are capable of understanding, reasoning, and generating human-like responses autonomously. Like, they understand the
environment and learn through it and take their steps based on the principle will provide the reason for which they have taken a certain step. Now moving ahead, why generative AI agents matter a lot. So there are three key points to
consider. Firstly, efficiency. They automate repetitive tasks with minimal human intervention. Next is high scalability. They handle large volumes of work, example customer support, content generation, and many more. And
finally, intelligence. They can adapt to different use cases and dynamically. Nobody has to write rules for them. Now let's focus on OpenAI and its latest release of agents. So OpenAI just released two new groundbreaking agents,
the operator agent and deep research agent. These two releases are absolutely massive for all AI agent developers. So operator is the first AI agent released operator is the first AI agent released by OpenAI. And what it does is it mimics
human actions on your browser. So it means it can scroll, type, click, and navigate Google web pages for you just like any other human would. But more importantly, and what many people have missed out is this agent is actually
based on a completely new model called CUA, which stands for computer using agent. So this agent, unlike standard GPT models, wasn't trained to just
output text. You like write a prompt and expect an output in the text format. Instead of that, it can go beyond it. This agent was actually trained to output mouse and keyboard clicks, which is exactly what makes agent so powerful
is exactly what makes agent so powerful in your browser. But not only that, it can also reason about every single action. This is also something that we haven't seen before. This agent doesn't just perform actions in your browser, it
actually reasons about every single step and what it should do next. Just like we discussed before about the generic overview of generative AI agents, same as the Open AI operator here. It takes the step based on the environment, and
also it will give you a particular reason for which or based on which it took a certain action, and it also asks you what to do next. Let's say if you want to book a ticket, it'll navigate to all the websites, it go through some uh
better flight options for you, let's say the low-cost ones, the low uh hours of flight ones, and best timings, and then finally selects the best one for you based on your recommendations, and asks you if it should proceed with
and asks you if it should proceed with booking or not. So, that's how it works. limitations as well. The first limitation is it has still a lower bit of accuracy. It's really high compared to other projects that we have seen in
the past where people just prompt the GPT models to perform actions in the browser. It can browse web using GPT-4 or vision capabilities, but it often whenever there was a model on a web page, like some other random model
available working readily on a web page, it would just accidentally pop up, and it would not be able to do anything because it would just get confused. It would not be able to close it, and then basically you would have to get
and close this model manually. And the second limitation is high cost. Right now, this particular operator agent is only available on $200 plus plan. So,
we can probably guess it's going to be pretty expensive in the API. Also, I do think that if you are focusing on the right thing, then costs are not going to Now, let's talk about the second agent that Open AI deployed. Now, the second
agent which was deployed by Open AI is called as Deep Research. So, this second agent released by Open AI was trained to perform comprehensive research. It can search the web. It can pull up the necessary resources and then it can
compile all that information into a really comprehensive document or maybe report as well. The groundbreaking thing about this agent is it's powered by the new O3 model. So, again, similar to the operator agent, but it doesn't just
search the web and provide you with results. It actually reasons about every single source and then it thinks about what other information it needs to find out next and it's really incredibly powerful. Deep Research doesn't just
internet just like some other standard GPT models. It would compile novel insights. Now, let's move on to the real-world applications. There are a wide variety of real-world applications. First one is customer support AI agents
which helps in automating help desk responses. Next, powerful AI systems like co-pilots for professionals. Third one is research and analysis agents for data summarization, literature reviews, and many more.
Fourth one is creative writing and content generation, automating social media posts, emails, etc. And lastly, AI-driven code assistants like GitHub co-pilot, Open AI codex, and many more. Now, let's go through the next part
which is core components of these generative AI agents. So, the foundation models are GPT-4, DALL-E, and Whisper. GPT-4 for natural
language processing, DALL-E for image generation, and Whisper for speech-to-text conversion. The next one would be prompt engineering and context So, here, designing effective prompts for AI agents to generate relevant
responses, using few-shot and zero-shot learning for adaptability, and lastly, implementing context-aware AI agents that remember past interactions from the user. Now, next would be the memory and long-term context storage. AI agents
typically process short-term prompts, but memory-based approaches, for example, LangChain's memory module, enable long-term reasoning as well. Now, storing and retrieving user-specific data for personal responses is also one
of the priority. Next will be the tool use and API integrations. Using external API to fetch real-time data, for example, weather, stock prices, news, etc., and integrating databases for structured information retrieval. And
lastly, further, we have autonomous decision-making. AI agents evaluate multiple options before making a decision. Planning and reasoning techniques to enhance responses is a priority for them. Proceeding ahead, we
will discuss how generative AI agents work. Starting with the first step, input processing and context awareness. AI agents analyze user input, detect intent, and extract entities. For
example, in a travel booking AI agent, it identifies destination, date, and budget from user queries, just like the example we discussed before, right? You provide the point A and point B, the best time for you to fly, and the best
number of hours that you focus on flying, and lastly, the destination, right? All these things will be evaluated in real time, the cost, the number of hours, the flight, the options provided, the amenities
things, and also budget as well. And based on that, they take a decision. Now, proceeding ahead, generating intelligent responses. So, here RAG, or also known as retrieval-augmented generation, AI AI external databases for
responding. Next, we have chain of thoughts reasoning or also known as COT reasoning. And AI breaks down problems into fine steps. And last step is fine-tuning the models. Custom-trained AI for specific industries. For example,
if you're preparing an AI agent for a dedicated sector like healthcare, legal, customer support, based on that, you will fine-tune your AI agent or model. Now, next is decision trees and multi-step reasoning. AI can simulate a
thought process, breaking down complex questions into manageable steps. For questions into manageable steps. For example, whenever you open Deep Seek or whenever you open OpenAI ChatGPT for the most premium version, you will write a
prompt and there will be an option called think, right? If you just click on that, it will explain you the thought process it is going through and how is it planning on giving a reason for you, right? And followed by that, we have
handling uncertainty and error correction. AI self-corrects through loops. Last step in this particular stage is fine-tuning and reinforcement learning or also known as RLHF.
Human-in-the-loop training to improve responses and AI learns from real-world interactions and adapts over time. So, it's basically called over time learning where it adapts to the real-world environment and learns things and
provides the best feasible solution. Now, the next stage is architecture and Now, the next stage is architecture and deployment of AI agents. So, in the first one, we have standalone versus multi-agent systems.
So, firstly, the standalone AI agents. They perform tasks independently. For They perform tasks independently. For example, ChatGPT. Multi-agent systems, in the other hand, work together to achieve a complex task. For example, one
agent for summarization, another for execution. Now, next in this particular stage is agent orchestrated model. A central AI agent coordinates multiple AI sub agents. For example, in an AI powered
research assistant, one agent gathers all the data that is required, the second agent summarizes all the information collected, and the third agent formats the output. Followed by that, we have cloud-based AI
Followed by that, we have cloud-based AI agents and edge-based AI agents. Cloud-based AI agents compute. For example, OpenAI API, Azure AI. And edge-based AI agents run on local devices for faster processing. For
example, Tesla's AI in self-driving cars. Followed by that, we have using APIs to build AI agents. OpenAI's API for GPT-4 powered by AI agents, LangChain framework for AI agent orchestration,
and LlamaIndex for document-based AI agents. The next stage we will discuss about enhancing AI agents with external tools. In that, the first step is integrating with databases. AI agents can query SQL
and non-SQL query databases for structured data retrieval. Proceeding ahead, we have connecting with web scrapers and APIs. AI agents can fetch real-time data using tools like Beautiful Soup, Scrapy, and OpenAI
plugins. Next ahead, using vector databases for context recalling. FAISS, ChromaDB, store long-term memory for AI agents. For example, legal AI agents retrieve case law from vector
embeddings. Lastly, we have autonomous agents and workflows. Auto GPT and Baby AGI are best examples. These AI agents that self-improve by learning from previous tasks. And Crew AI, which is a set of
multiple AI agents collaborating to solve a complex problem. Now, the main part, challenges and limitations of generative AI agents. In that, the first one will be the ethical considerations. AI bias and
fairness concerns are the primary ones. Addressing AI hallucinations, which is false information generation, and then we have computational cost and efficiency, which is a major concern amongst all the industries. Large AI
models require significant computational power. Strategies to optimize AI agent efficiency will also considered to be expensive to hire many training models or also human programmers. Followed by that, we have security risks
and data privacy. Risks of data leaks or data breaches and AI jailbreaking is one of the highest security threat. Implementing secure AI architecture is need to focus on. And lastly, we have legal and compliance
issues. This would major be concerned from the enterprise level. GDPR, HIPAA, and AI regulations compliance is mandatory, and AI liability in mandatory, and AI liability in decision-making is also important. And
discussion, which is future trends and advancements in AI agents. Starting with the first one, AI agents with emotional intelligence or also known as affective AI is capable of detecting emotions and adapting responses accordingly. For
example, if you have designed a therapeutic AI chatbot for mental health. Now, the next one, self-improving AI systems. So, meta-learning and auto ML are a few examples. AI agents that learn
from new data autonomously. For example, AI adjusting marketing campaigns based on customer responses. And now, the next one, AGI and roadmap to superintelligence. Well, this always ends up with question. Will AI agents
evolve into AGI, or also known as artificial general intelligence? And are the implications of self-learning AI real? This is the major question. And lastly, the role of AI agents in the future of work. Now, we can see there
are a lot of AI agents like Copilot, which will assist humans in their day-to-day task, whether it is business, health care, or education. And also, you have a lot of automation and transformation happening in the IT
industry, which will reduce the work hours and stress on the employees by automating all those mundane tasks by the help of Copilots, also the generative AI agents. &gt;&gt; Welcome to the short course on
generative AI models for beginners by Simply Learn. So, in this course, we will learn about the basic understanding of generative AI. Then, we will move on to the see applications of the many GenAI tools.
And thirdly, we will understand the GenAI models, and then we will create our first GenAI model. And then, we will move on and see the real-world impact of the generative AI models.
And then, we will create a chatbot using the GenAI model. And then, we will conclude the course with the limitations of the GenAI models, and finally, we of the GenAI models, and finally, we will see the future of GenAI models.
Now, let us begin with the first chapter, that is basic understanding of generative AI. So, let us begin with what is generative So, generative AI is a type of artificial intelligence that creates new
artificial intelligence that creates new content, such as text, images, music, or content, such as text, images, music, or even code from scratch based on what it has learned. So, unlike the traditional AI, which just analyzes or classifies
data, like recognizing faces or recommending products, generative AI creates. It's like the difference between a judge and an artist. One evaluates and the other one imagines and produces.
So, let's break it down with a simple real-world examples. The first one is real-world examples. The first one is ChatGPT, that is a text generation. So, what it does? So, ChatGPT, like the one you're talking to right now, generates
you're talking to right now, generates human-like conversations and answers. So, you can ask it to write stories, emails, explain difficult topics, or emails, explain difficult topics, or even debug a code. For example, you say,
"Write me a bedtime story about a robot and a unicorn." The ChatGPT creates an entire story for you on that particular spot. So, let's move on and see DALL-E, that is for the image generation.
So, DALL-E creates pictures from words, that is the text prompts. Then, artists and marketers use it to Then, artists and marketers use it to generate logos, illustrations, or more
without hiring a designer. For example, you type a cat wearing a For example, you type a cat wearing a spacesuit on Mars, and DALL-E draws it instantly. Now, moving on to GPT-3, that is text plus logic task. So, what it
is text plus logic task. So, what it does? So, GPT-3 is a powerful language model that can create and write essays, summary books, translate languages, and even write computer code. So, developers use it in apps to create smart features,
like the AI writing assistant. So, then you give GPT-3 a blog title and it writes the full article with an intro, subheadings, and conclusion. So, after knowing all this, let's move on and see why are generative models. So,
generative models are changing the way we create content across multiple fields. So, under the text field, we are writing books, articles, reports, or even poetry.
And then we are assisting in the customer support with the chatbots. And then under the images section, we are helping the artists generate concepts and visuals. And then we are even handling the
speeding up design and marketing content creation. And then under the music part, we are composing the background scores for videos or games. And then we are helping musicians brainstorm the melodies or the lyrics.
And then finally, we write the boilerplate code under the coding section, debugging, or even creating full web apps. So, now moving on, we'll full web apps. So, now moving on, we'll see traditional AI versus generative AI.
So, let's compare both side by side with a simple explanations. So, the main goal of traditional AI is to analyze and make decisions. And the main goal of generative AI is create new content.
For example, under traditional AI, predicting stock prices or recommending books on movies. And in the generative AI, we are writing stories, generating images, and so on. So, traditional AI handles data, it uses data to classify
or forecast, and generative AI learns from data to generate something new. The traditional AI, it labels, scores, and categories as output, and generative AI, the text, images, music, videos, and so on. For example, traditional AI is face
recognition, fraud detection, spam filters, and generative AI is ChatGPT, DALL-E, MusicLM, and so on. So, down the line in the next chapter, we will see
the applications of some generative AI tools like the ChatGPT, DALL-E, and GPT-3. So, imagine this. You are working on a college project. It's late at night, and the deadlines is tomorrow. You need a well-written report and a
poster design and a clear presentation. So, total chaos, right? But what if I tell you that there's an AI assistant that can help you with all of this in just minutes. So, welcome to the world of generative AI. Today, we'll explore
how tools like ChatGPT, DALL-E, and GPT-3 are being used by the students, content creators, and professionals to do amazing things faster, easier, and smarter. So, let's break down each tool and see how you can actually use them in
and see how you can actually use them in real life. So, this is ChatGPT-4o. If you're not aware of what ChatGPT is, so ChatGPT is an AI chatbot that can create, write essays, summaries, articles, explain topics, solve coding
problems, and even conduct mock interviews all through one single chat. it. So, ChatGPT model, it uses this GPT-4,
the latest in the generative AI pre-trained transformer that is a GPT pre-trained transformer that is a GPT family. So, GPT-4 is designed for more nuanced and human-like conversation. So, have a look at the real ways we can use
it. So, the first prompt we are going to try is explain the greenhouse
like I am in class 12. the greenhouse effect explanation, and
it is asking the ChatGPT to behave that it is explaining to a class 12 person. it is explaining to a class 12 person. So, we will give this.
generation. Just like here the greenhouse effect is a natural process that warms the earth's surface. It happens when certain gases in the earth's atmosphere trap the heat. Here's how it works. It will have a detailed
explanation on the solar energy earth surface absorption, greenhouse gases, heat trapping, and all. So, after all these four points, it has also given without the greenhouse effect, the earth would be too cold to sustain life.
However, human activities like as burning fossil fuels, deforestation, and industrial processes. So, you can basically think it has given one nutshell. And if you just understand this, you have a clear vision of what a
greenhouse effect is. So, the first point we can see over here is that it is like a own tutor. So, the first point we have just been to Chat GPT that is founded by chat.openai.com. We have typed a question and we have got
a simple and easy to understand answer. And even you can have a follow-up And even you can have a follow-up question. Like if I ask, is solar energy related to
effect? You can even have a follow-up question. Yes, solar energy is related to the greenhouse effect. So, it's almost like it's own tutor. So, great for the late-night cramming or
the quick concept clarity. So, this is it. And now moving on to the second thing that it can help with, that writing assignments or essays in just
minutes. So, we will ask Chat GPT, write a 350 word essay on
climate change for for college purpose. conditions. You can even customize it by saying make it simple and yet
and awesome. So, it will basically create it has even created a one title, a growing concern Climate change it has created paragraphs and basically you can even customize
this word count and topic then review the generated draft. You can even the generated draft. You can even personalize it to apply this personality and done and dusted. You can just have this. Now writing feels less stressful
and it would be more fun. It can even give you an interview experience. For example, just imagine you are having your interview on the next Monday and you are preparing and you are very confused about what to prepare and what
not. Then you can basically ask Chat GPT to behave like a interviewer for a particular job. I can even paste your job description from that particular company and you can ask that what are
the probable questions I can have. So, let's have a demo. Ask me three HR questions
for a software internship and suggest
and suggest improvements. And in no time it has started generating. Here are the three HR internship. The first one, can you walk me through a recent coding project you
have worked on and explain how you approach solving the problem? And you know what? This is a very very common question and it has given one tip also, suggestion. This question evaluates your problem-solving skills, your technical
ability and how you approach development task. It gives the interviewer insight into your practical experience and thought process. So, how do you manage multiple projects or the task simultaneously? So, this question
assesses your time management skills and ability to prioritize the task. The software development often involves juggling multiple assignments. So, this gives the interviewer an idea of your organization skills. So, this is very
important. It has also written, how do you keep yourself updated with the latest programming languages and technologies? So, basically this technologies? So, basically this question looks after your own learning
So, the self-improvement part. So, it has even created one and answer over it. And it has also suggestions for the improvement, provide specific examples, highlight the soft skills, prepare for the problem-solving questions and so on.
And if I for example, if I write for example, if I write customize it with a data analyst
fresher role having having no experience.
And it will give you answer in no it will give you answer in no time. career as a data analyst? And what are the steps you have taken to prepare
yourself for this role? Can you describe a situation where you had to analyze data or solve a problem using numbers or data in your studies or personal projects? How do you approach learning new tools or technologies, especially
when it comes to data analysis software? And then we have this technical question. What is the difference between the structured and unstructured data? What is data normalization? How comfortable are you with the data
visualization tools? Can you describe a visualization you would use to represent sales data over time? Can you explain what a pivot table is and where it is used in data analysis? And there are suggestions that you are showcase your
learning journey, focus on the problem-solving skills, prepare for the basic technical questions. Since you are a fresher, the technical questions may focus on the fundamentals like the Excel, Python,
that is the libraries, Pandas, and so on. So, in no time it can even have a mock interview over here. So, you can even create ideas over here. It can brainstorm you with the ideas available.
For example, you have a group project and you want the idea. I'll type over here, give five creative ideas for a
in no time it has created these five ideas. Sustainable packaging design challenge. Objectives design a sustainable It has created one objective. It has given one approach for you and one outcome. Then
it has created community-based urban garden. And for each topic, if you see, for each idea, if you see, it has created the objective, the approach, and So, these projects can not only be educational, but also impactful. So, you
never run out of inspiration from ChatGPT. So, ChatGPT is a lot. And here, ChatGPT. So, ChatGPT is a lot. And here, if you can actually see, it has various search, deep search, and create image. And you can basically do everything you
want from ChatGPT. Let's now move on to DALL-E. The next we would be seeing is DALL-E. So, DALL-E is an AI that creates images from text. You tell it what to draw, and it generates the artwork,
logos, and visuals. So, you would be getting DALL-E from this ChatGPT only. And in the explore GPTs, in the sidebar, you have to just mention DALL-E
2. So, there are various versions of DALL-E. And we'll start chatting. And now, if you close this sidebar, you can basically see
DALL-E 2. So, DALL-E is an AI that creates the images. And DALL-E, this model, uses a version of the CLIP model. That is a contrastive language image That is a contrastive language image pre-training. Combined with the VQ-VAE 2
or other generative image models. So, DALL-E creates images from the text descriptions, generating highly creative visuals based on the natural language visuals based on the natural language input. So, let's see how you can use it.
for example, we you're going to design one of the college poster design. So, instantly it would be done. we would write create a poster
for a college fest DJs, {comma} neon lights,
and a
And we can basically give the prompt. And in no time, it will create one great image for you. You can write the And you can see it has You can write the And you can see it has created a great a great image.
You can basically download this image and do anything. featuring a vibrant DJ scene with the neon lights and the energetic crowd.
So, forget Photoshop. Let AI do the heavy lifting. So, then you can create presentations over here adding better visuals. over here adding better visuals. So, I'll write draw a cartoon
So, I'll write draw a cartoon showing
in a river. And Now, you can insert it into the PowerPoint or Canva and design it on
your your own. It acts as a vector, it acts as icon, and anything you want to Then you can instantly create club logos and branding. So, you can even generate a
logo for over here through DALL-E 2. Logo for a coffee club
books and steaming and steaming mugs.
logo and it feels like real. So, you have just given one prompt and in and get four to five variations of logo. You can even ask it for a different version. So, great for the student clubs or college
Then, you can even have it for the costume design or any ideas again. So, we'll try and futuristic
design for a for a sci-
with glowing elements.
it has created a futuristic costume design for a scientific fiction play with the glowing elements. And it is very amazing. The qualities of the very amazing. The qualities of the images are of high quality.
detailed prompts and you will get a great great output from DALL-E 2. art students and theater clubs, this is your new secret weapon. So, let's move on to GPT-3. Our next GenAI tool is GPT-3. So, GPT-3 is the
powerhouse engine behind the tools like ChatGPT. So, it doesn't just chat, it can write full code, research summary papers, business emails, and more. So, papers, business emails, and more. So, GPT-3 is a predecessor of GPT-4 based on
the same GPT architecture, but with a 175 billion parameters. It was one of the most advanced language models when it was released. So, let's explore how it helps in the real-world tasks. So,
initially, we can use ChatGPT because it is a ChatGPT 4 version. There is a GPT-3 is basically a predecessor of GPT-4. So, I'm using ChatGPT over here. You can also use ChatGPT 4 to understand the GPT-3. So, the first one is building
websites without coding. So, you can even write write a HTML for personal
portfolio with my name, and contact form.
You can just copy it, paste it in VS Code, or any of the code editors you proper output of this.
Then, you can even summarize a lengthy research paper. You can have to just upload the paper, and if you see this plus section, you can just
click on this, you can just upload from your computer, and summarize it in 2 seconds. It can even brainstorm you with the project and the startup ideas like suggest
app ideas for for college students.
Now, you can see the first one is study buddy finder. Help students find partners based on their courses, schedules, and many preferences. And you can even see campus event hub, course material exchange, mental wellness
material exchange, mental wellness check-in, foodie finder, and so on. It has even created bullet points for a better understanding. So, it is a noise-free study timer. Use it to kickstart your next hackathon or
competition or anything. So, I'm not dragging it more because as I have already showed you the use of ChatGPT. So, that's it. Moving on to the next lesson of understanding the generative AI models.
generative AI models. Imagine you are teaching a computer to be creative, like painting a picture, writing a poetry, or designing a dress. That's exactly what generative AI models aim to do.
So, learn from the data and generate new content that looks, sounds, or feels content that looks, sounds, or feels real. But, how do this pull off? Let's break it down into the three major families of models. The first one is
families of models. The first one is GANs, that is the generative adversarial networks. Then the second one we have is VAEs, that is the variational autoencoders. And the third one, the final one is
transformers, like the GPT. So, let's understand this in a detailed one. The first one is generative adversarial networks, GANs. The artist versus the critic game.
So, how it works. Think of GANs like a competition between the two neural networks. The generator is like a rookie artist trying to create a fake art. The discriminator is the tough critic trying to tell the real art
from the fake. So, the artist creates random images. So, the artist creates random images. The critic says, "Fake." or real, and the artist learns to get better. So, over the time, the fake images become
indistinguishable from the real ones. So, there's a real-world analogy. It's like a student that is a generator forging a a student that is a generator forging a signature and a teacher discriminator
spotting it. As both improve, the forgeries become nearly perfect. So, there are certain applications. The first application is creating realistic human faces, then style the transfer in the art, and then we have
fashion design prototypes, and then finally, we have deep fakes that is controversial but technically fascinating. challenges. So,
instability. Then, the second one is it requires a lot of data and computing. So, moving on, we'll understand variational autoencoders, that is the VAEs. The data compressor with
creativity. So, VAEs are like the compress and the rebuild systems. They take input data like an image, compress it into a low-dimensional space that is called a latent space, and then
called a latent space, and then reconstruct it. But, here's the twist. The VAEs don't just remember, they imagine. So, by sampling different points in the latent space, they can generate new similar looking content. So
let's move on and see the applications. First application is anomaly detection. So what doesn't fit in the normal data? Then the second one is generating
handwritten digits and faces. And then we have data compression. And then finally we can see drug discovery, that is generating molecular structures. So now let's move on to the strengths. So more stable than the GANs, and it
gives a well-structured latent space for the interpolation. So now moving on to the third model, we have transformer models, that is a master of the So unlike the GANs or the VAEs, which focus
more on images, transformers shine in handling sequences, specially the text. The core data idea is attention mechanism, a way to let the model focus on the different words when generating a sentence. So for example, when GPT
generates a sentence I like ice cream because it's sweet, it pays attention to the context, understanding how because relates to the ice cream and sweet. So let's now move on to the applications. The first one is chatbots,
like the ChatGPT. And then we have code generation, that is the GitHub Copilot. And then thirdly we have language translation, and finally we have summarization and content creation. And then let's move on and see why
transformers stand out. So the first one is they scale data well. The second one is they can be fine-tuned on specific tasks, like the medical summarization or legal drafting. Then finally we have the handle, the long-range dependencies
better than the older models like the RNNs or the LSTMs. Then, we have a comparison and the use cases. So, the first one is GANs, and it is
best at image generation. And then, we have an example use, that is the face synthesis. And the associated strength is high realism, and the weakness is it is hard to train. Then, we are seeing the VAEs, and it is
best at data compression, plus it is also good in anomaly detection. So, it is used as a latent space exploration, and then coming to the strength, it is very stable training, and then coming to weakness, it has a
less sharp outputs. So, then coming to the model type, that is the transformers, and it is best at text and language. And then, we have an example use, that is a chat GPT and summarization. And then, we have
strength, that is powerful with a large data. And then, we have certain weakness, that is expensive to train. Now, moving on, let's clear something. Which to use and when? So, if you don't
know which generative model to use for what, it can mess things up fast. So, let's say you are building an app that needs to write emails, make pictures, answer questions, and turn speech into text. If you use the wrong model for the
wrong job, your results will be poor, things will be slow, and you will waste time. So, let's understand which models performs what. So, for a sharp realistic images, we are using GANs. And then,
coming to the next, for variation and latent space analysis, we are using VAEs. And then, finally, for the language or the text-based tasks, we are using transformers. So, finally, how do these models
actually work? So, the GAN training process is the first process is generator makes the fake data. Then, the step two is the discriminator tries to classify the real versus fake. Then, the third step is both get better at their
jobs with each round. So, the final goal is to fool the discriminator with the realistic outputs. So, the transformers under the hood pays attention mechanism. So, it helps model understand which words matter more. Then, we have
self-attention that is allows the model to look at all the words in a sentence at once. And then, we have positional that is encoding and it helps track the orders of the words as transformers don't read left to right like humans.
So, this is why models like GPT-4 can summarize, translate, and generate and text so well. So, understanding the differences between these generative models is like knowing which tool to use for which creative
job. So, GANs are your digital painters, VAEs are the smart compressors with imagination, and transformers are your text wizards. So, down the line, we'll how to create our first generative AI model.
In this chapter, we will be creating our first generative AI model. So, the objective of this lesson is breaking down the steps of building a generative AI model using a pre-trained model. So, we are here not training a
model from the scratch. So, instead, we will fine-tune or directly use a pre-trained model. So, this makes it fast and beginner-friendly. So, if I tell the steps in a nutshell, it will be we will be first choosing a
pre-trained model. For example, GPT-2 because it is lightweight and good for beginners. The source from which we will getting the model is hugging faces model hub, and then we will be setting up the
see. Then we will be installing the libraries, the transformers, torch, and then loading the pre-trained model and the tokenizer. Then we will be writing a
prompt and generating a text. So, here we will be using the pipeline API for simplicity and parameters like max length and temperature.
So, let's begin. So, here I am inside this Google Colab and I have chosen a new notebook to begin with, and then I'll just insert pip install
And shift enter. And may It may take some time. installing all the required transformers. pip install means install this package in Colab, and transformers is a library
by the hugging face. So, it gives us access to the powerful AI models like GPT-2, BERT, and more. And then we will see
And then we will see we will install from transformers pipeline. And then
we will be using the loading generator GPT-2 text generator. So, generator and tap.
and tap. So, And basically, this pipeline is a shortcut tool. It hides the all the complex code needed to run the AI models. So, here the text
generation tells like, "Hey, I want a model that can write text for me." And GPT-2 is the actual model name. It was trained by the OpenAI to predict what word comes next in a sentence. And then basically,
you can see that it has model 7 tensors, 100% has loaded everything. And and and the storage is also enabled for this repository. And then we'll begin with the prompt, okay?
So, let's give the prompt. And let's keep it very simple. Prompt equal to once upon a time once upon a time in a world
And then in the next line, I will be in the next line, I will be writing result is equal to generator.
inside it. And then we'll be giving the max length one parameter. And I will be specifying it as 100. And I will be giving the number
will be giving the number return sequence.
And I don't really require this. So, I will be directly printing the result. I will be directly printing the result. The first one, the first doc will print
we'll be passing the generated the generated text.
a starting sentence you give to the AI. Like saying, "Hey, start a story with this and this and this." So, max length is equal to 100. So, this particularly means generate up to 100 words or characters after the prompt.
And then you can even see I have given this num return sequence is equal to one. This is asking for just one version of the result. So, you can basically ask for two, three versions. I have just asked for only one. And then the result,
And in the third bracket I have given a zero and the generated text. It extracts the generated text from the result. So, you will see the AI write a continuation of your prompt basically. So, that's your first generative AI output. I can
basically see, "Once upon a time in the world of AI, your only limit was when you could kill monsters, but now you can for the first time ever." The new monsters are in the battle in the blood of elves now. Not the usual black and
white battles that we You have experienced with the past couple of experienced with the past couple of years. And it has created a great one. So, you can even see the power of generative AI. It has the power of
completing your text, your prompt. So, then let's write result equal to generator.
enter then max length I will again mention then enter I want the num
num return
and have to put a comma over here, okay? So, don't forget this, guys.
top key. Don't worry, I'll explain one by key. Don't worry, I'll explain one by one as I write.
equal to 0.95 comma then I'll write do samples
equal to true. Basically, a Boolean value. So, then I'll after this I'll print the result. Print result
and in third brackets with the first one and then again third brackets generated text
the basically going to shift enter. we have parameters like temperature. So, temperature, what it does, it controls
the creativity. This is the lower is equal to safe, and higher is the wild, equal to safe, and higher is the wild, and creative is 0.7 medium. So, this top k top {underscore} k is equal
to 50. So, pick from the top 50 likely next words instead of just the top one. Then, we have this top {underscore} p is equal to zero. It is the nuclear sampling. That is a more natural randomness. Then, we have do sample is
equal to true. That is enables the randomness. So, without this, the output is too predictable. So, once you run this code, you will see something like once upon a time in a world full of AI, and with every
advancement in AI machines, there has been a change in the way we think about the world. And we need to make sure that we can make sure that we can make sure and it has repeated it so many times. So, every time you run it, you might get
a slightly different result. So, over here, you have used the hugging faces here, you have used the hugging faces transformers library to load the GPT-2. Then, you wrote a prompt, and then the AI continued it using its training. So,
you called it. So, you did it and without any complex machine learning setup inside the Google Colab. So, this is your very first generative AI model. In the next chapter, we'll be seeing the real world of impact of
generative AI models. So, generative AI is not just about another buzzword. It's fundamentally changing how businesses operate, solve problems, and serve customers. From generating product descriptions to
designing aircraft parts, generating AI is stepping into roles that once required human creativity and intelligence. So, in this session, we will explore how industries across the board are using
generative AI in meaningful and practical ways. The first one is healthcare. That is a faster diagnosis and drug discovery. So, in healthcare, generative AI is helping doctors and scientists make faster decisions.
scientists make faster decisions. The first one is medical imaging. So, AI can analyze medical images like the X-rays or MRIs and create reports to help doctors quickly identify the problems. For example, a hospital uses
problems. For example, a hospital uses AI to write reports for doctors, cutting down the time spent on each patient. Then, we have drug discovery. So, AI helps scientists discover new medicines faster by predicting how proteins fold,
which is important for the creating drugs. For example, a company used AI to drugs. For example, a company used AI to discover a new drug in just 18 months, discover a new drug in just 18 months, which usually takes much longer time.
Then, moving on to the next one, we have finance. That is a smarter risk management and report generation. So, in finance, AI is making it easier to analyze data and detect the problems. The first one is automated financial
reports. So, AI can create financial reports based on data, saving time for analysts. For example, a hedge fund uses AI to write the market reports in
Then, we have fraud detection. So, AI helps banks detect fraud by generating the fake transaction data to test their security systems, making them more reliable. So, for instance, MasterCard uses AI to create realistic transaction
data to improve their fraud detection system. So, now moving on to the third we have retail and that is the personalized shopping experience. So, retailers use AI to create personalized shopping
experiences for customers. The first one we have is a product descriptions. So, AI writes product descriptions that are customized for different customer groups
such as casual descriptions for young buyers and formal ones for the professionals. For example, a clothing brand uses AI to automatically write descriptions for thousands of products. And then we have virtual try-ons. So, AI
lets customers try on clothes or makeup virtually to see how it would look on them before buying. So, for instance, Sephora's app uses AI to suggest makeup
and lets users see how it looks on their face using their phone camera. So, now moving on to the fourth one we have manufacturing. This is a faster have manufacturing. This is a faster design and maintenance. So, AI is also
making manufacturing more efficient. The first one is AI generated designs. AI helps create lighter, stronger parts for products reducing waste and material
products reducing waste and material cost. For example, an airplane company used AI to design parts that were 50% lighter but just as strong as a traditional designs. Then moving on we have predictive
maintenance. So, AI helps factories predict when machines will need maintenance reducing downtime and improving efficiency. For example, a factory uses AI to automatically create maintenance schedules for machines based
on their performance data. So now moving on to the fifth one we have education. That is a personalized learning for students. So AI is making education more personalized and accessible.
The first one is AI tutors. So AI powered tutors can help students with the homework by guiding them step-by-step through problems. For example, a student stuck on a math problem can get hints and explanations
problem can get hints and explanations from an AI tutor available at any time. Then we have automated feedback. So teachers use AI to create quizzes and provide feedback on assignments saving them hours each week. For instance, a
teacher uses AI to review essays and provide comments in just minutes instead of hours. So now moving on we have sixth one is entertainment that is creating at
a scale. So in entertainment AI is being used to create content faster and cheaper. First one is script writing. AI can help writers come up with ideas, generate dialogues, and even create storyboards before filming. For example,
a filmmaker uses AI to plan scenes and create visuals before they start shooting save time in the pre-production. So then we have virtual characters. So AI can create virtual characters that speak and act which is
useful for movies, games, and social media content. For instance, AI is used to create virtual influencers who interact with their followers on social So now moving on down the line we will see how to create a chatbot in Google
see how to create a chatbot in Google Colab. So before we dive into the code, let's first understand what the hugging face transformers library is. So, the hugging face Transformers library is one of the most popular libraries for the
natural language processing, and it provides the pre-trained models that can perform a variety of tasks like the text generation, text classification, named entity recognition, and more. So, here in the code, we will be using a
model called DialoGPT-Small, which is a variant of GPT-2 designed for conversation generation. So, the DialoGPT was trained on the large conversational datasets, making it a perfect fit for building the chatbots.
So, let's now build the setting up the environment. So, for this, we will environment. So, for this, we will install pip install install pip install transformers.
transformers library. You need this to work with the pre-trained models provided by the hugging face. So, this will allow you to load the models like the DialoGPT we will be using here, a conversational AI model,
responses. And then, and here we will begin with importing the necessary libraries. From transformers, import the auto model for the casual LM auto tokenizer, import torch, and then we will run this. So,
here we are importing the necessary components like the auto model ID for the casual LM. So, this class basically loads a pre-trained model for the casual language modeling, which is used for the text generation task.
So, we will use it to load the DialoGPT model. And the auto tokenizer, the tokenizer converts the text into tokens, tokens which are known as numbers. Now, the the can process and then converts the
model's output back into the human-readable text. So, the torch is then PyTorch is a deep learning framework used by the model. It's used for the tensor manipulations as models rely on these tensors for the
input and output. So, let's begin. Then we will begin with a loading the model and the tokenizer. So, here this converts the raw text that is the user input into tokens.
word or phrase is mapped into the unique ID making it easier for the model to process and generate a response. So, then model is that pre-trained dialo GPT small model is loaded over here. So, this model has been trained on a large
corpus of dialogues, which enables it to generate the relevant responses in a conversational manner. So, then we will begin with the chat initialization. And before beginning with the chat initialization, we will see that the
tokenizer config.json is 100% complete. And the best part of Colab is it will give you all the status of how it is complete and for which particular merges.txt or vocab.json which is completing and
what problem is there. So, let's just run this. let me explain. So, chat history IDs. So, this is a variable where we store the history of the conversation. So, this helps the
model maintain the context across multiple user inputs. Then we have we're having a print statement to start chatting with your AI. Type quit to exit. So, this provides a welcoming message to the user to start chatting
and gives the exit command. So then we will begin with a chat loop that is the main logic of the chatbot. So then we'll begin with this chat loop logic. So we have written for step in
range five, user input is equal to input and then you and then semicolon and then a space. And then we have written if user input.lower is equal to equal to quit then I have to break it.
Break the loop. Break. So then I'll run this. And this section of the code basically uses input to get the user's message and exit condition. If the user types quit, the loop will
break and the chat ends. So let's see. And here basically you can see I have written here and still it doesn't break. So let's write quit over here and let's see what happens. So you can see that it has stopped generating.
Means it has got out of the logic. So here we have this encoding user input. So encoding the user input is converted into token IDs using the
tokenizer.dot.encode. So the EOS token that is the end of sequence token is added at the end of the user input. This tells the model that the input has ended and it prepares for generating a response. Then tensors.
So this is a return_tensors equal to PT. That part ensures that the equal to PT. That part ensures that the encoded input is returned as a PyTorch tensor. This is the format that the model expects. So we will run this.
And then we will be moving on to the managing the conversation in history. So here we are having this managing conversation history and it is up to maintaining the context. So we want to keep the track of the the
history so that the model can respond in context. So, if there's previous chat history, that is a chat history IDs, it is concatenated with the new input, allowing the model to consider the
entire conversation so far. So, if there is no previous history, that is the is no previous history, that is the first user input, the new input IDs are used directly. So, then we will be moving on to
generating a response. So, here we are concentrating on a response. Chat history IDs is equal to model generated bought input IDs max length equal to 1,000 and bought token
length equal to 1,000 and bought token IDs equal to tokenizer.eos_token_id. the concatenated input to the model, that is model.generate, and generates a
response. The maximum length specifies the maximum length of the generated text. So, in this case, this set to 1,000 tokens. Then we have bought token ID, that is it ensures the padding tokens are handled
correctly, that is the model needs to know which tokens are just padded. Then we will be moving on to the decoding the response. And here we are seeing response is equal to tokenizer.decode chat history IDs, that
is the bought input IDs.shape and everything. So, basically decoding this generated token IDs are converted back into the human readable text using the tokenizer.decode. Then we have skip the special tokens.
The skip special tokens is equal to true, it ensures that any special tokens, like the EOS, are not included in the final output. in the final output. So, we will run this.
printing the response. That is displaying the response. So, the print this generating the response is displayed to the user. So, we'll run this. And you can see, "Bot, I'm so sorry."
So, what happens when you run this code? You initiate the chat, and the bot welcomes you to start chatting. So, every time you input something, the bot takes your message and codes it and sends it to the model along with the
generates a response considering the entire conversation so far. And the bot outputs that response. So, this loop continues until you type quit, ending the
conversation. So, building a chatbot is essential because it enhances the user engagement by providing the real-time personalized interactions, making it more dynamic and the user-friendly compared to the static
content. So, chatbots are available 24/7, ensuring the users can get immediate responses at any time. So, by automating the repetitive tasks like answering common questions, processing orders, or providing basic customer
support, chatbots free up valuable resources, allowing human employees to resources, allowing human employees to focus on more complex tasks. So, then, additionally, chatbots can improve customer satisfaction by quickly
resolving the inquiries, reducing the wait times, and enhancing the overall user experience. So, implementing the context management allows the bot to remember the previous interactions, ensuring a continuous and more
personalized experience for the user. So, finally, enabling the multi-turn conversation handling means the chatbot can engage in back-and-forth dialogues rather than only answering the isolated questions, so making it feel more
conversational and human-like. So these features collectively help create an intelligent, efficient, and engaging chatbot that can significantly improve user interactions and streamline the business operations.
So you can even fine-tune your own model. So this could be your own data set to make it more specific to a particular domain or a use case. You can even make it worth handling the
longer conversations so that the it remembers the chat history for a long time. So guys, you have made your first chatbot and down the line, let's get improved.
in this lesson, we will understand about the limitations of generative AI models. So the first one is lack of understanding. So AI models generate contained by predicting what comes next based on the
patterns in data. But it don't truly understand the content. For example, when an AI creates an article or a code, it doesn't know what it's writing. It's just mixing patterns based on the past data.
data quality. So the AI's performance is only as good as the data it's trained on. So if it has biased or incomplete data,
it can generate biased or inaccurate results. Think about biased hiring algorithms or flawed health care recommendations. Real flawed health care recommendations. Real issues caused by the bad training data.
So now moving on to the third one we have creativity limitations. have creativity limitations. So AI can produce impressive creative outputs like artwork or music. So however, it struggles to innovate or
So however, it struggles to innovate or produce truly original ideas like a human artist would. It's more about remixing the existing content rather remixing the existing content rather than inventing something groundbreaking.
Now moving on to the next, we have ethical concerns. So AI models can sometimes generate harmful content. From offensive language to deep fakes
that are hard to distinguish from the real footage. real footage. So this poses risk for misinformation and ethical dilemmas, especially in media and entertainment.
So now moving on to the next, we have context and nuance. So generative AI can miss the subtle nuance in conversations or emotions. For instance, if you ask a model to help with the sensitive topics
like mental health, it might offer advice that's generic and more tailored advice that's generic and more tailored to the individual's emotional needs. Next we have cost and resource intensive.
So training a generative AI model requires huge amounts of computing requires huge amounts of computing power, time, and energy. So these models often run on the powerful servers consuming a lot of resources, which
raises concerns about the sustainability and the environmental impact. So now moving on to the next one, we have limited problem-solving capabilities. While generative AI is great at tasks
like writing essays or summarizing text, it's not well suited for the task requiring the deeper reasoning or the complex problem-solving. It can simulate
problem-solving, but it doesn't truly understand the solution in the way humans do. Now moving on to the next one, we have risk of over-reliance. So with AI generating more and more content, there's a risk of becoming too
dependent on the technology. So, people might rely on AI for the task that require human expertise or critical thinking, which could hinder the personal creativity or decision-making. So, now moving on we have data privacy
issues. So, AI models can sometimes generate content based on the personal data, raising concerns about privacy. So, this is specially problematic in the So, this is specially problematic in the areas like healthcare, where sensitive
data can be inadvertently shared or misused. So, now moving on to the next one, we have generalization challenges. So, generative models are great at specific tasks,
but struggle with the broader generalization. For instance, a model trained to generate images of the cats may struggle to create the convincing images of a completely different animal, showing that AI is limited in its
Now, moving on down the line, we'll understand the future of GenAI models. The first one is improved understanding and comprehension. So, the future of
generative AI will likely see models that can understand the context more deeply, providing more relevant and meaningful outputs. So, we might see AI models that can truly comprehend the
meaning behind the responses, rather than just spitting out the data. So, next we'll move on to the enhanced creativity. So, as AI models evolve, we
can expect them to push boundaries and become more innovative. So, with advancements in the techniques like the reinforcement learning and neural architecture search, we could see AI producing truly original content,
potentially revolutionizing fields like the art, music, and literature. So, now moving on, we have more ethical AI. So, in the future, we will likely
AI. So, in the future, we will likely see more regulations around AI usage to prevent the harmful outcomes. So, AI companies are already working on improving the transparency, reducing bias, and making their models more
ethical and responsible. So, now moving on, we have better So, now moving on, we have better personalization. So, the future holds AI that can offer more tailored personalized experience. So, imagine an
AI that truly understands your needs and preferences. So, providing you with the content, products, or services that feel products, or services that feel personalized and relevant.
Now, moving on to the next one, we have energy efficiency. So, as the demand for the generative AI grows, researchers are focusing on creating more energy-efficient models. We may see smaller, more powerful models that can
run on less computational power, reducing the environmental impact. So, reducing the environmental impact. So, next we have human-AI collaboration. So, rather than replacing humans, the future of generative AI models is likely to be
centered on collaboration. So, you will see AI helping people be more productive, augmenting creativity, and providing insights that humans can use to make better decisions. So, now moving on to the next, we have
integration with the new technologies. So, generative AI will continue to integrate with the new technologies like augmented reality, that is the AR, then the virtual reality, that is the VR, the blockchain, and and opening up new
possibilities for the immersive experiences and the decentralized applications. So, now moving on, we have solving complex real-world problems. So, in the near future, we could see AI models
working alongside scientists to tackle the big challenges like climate change, pandemics, and global food security. So, AI could help analyze data, predict trends, and offer innovative solutions to these pressing problems.
to these pressing problems. So, now we'll see the next one we have more accessible AI. So, as technology advances, AI will become more accessible. Tools will be easier to use, and people
from all the walks of life, not just the technical experts, will be able to harness the power of AI to create and innovate. So, while generative AI models are still evolving, their potential to transform the industries, enhance
creativity, and solve the complex problems is undeniable. But, it's important to address their limitations and work towards creating more ethical, efficient, and responsible AI systems. So, the future is exciting, and we are
just getting started. So, guys, congratulations on completing the GenAI models for beginners course by Simplilearn. You have now built a strong foundation on the generative AI models by learning what generative AI was, how
it impacted the real world, then creating chatbots, creating your first GenAI model, and then finally closing it with the future of GenAI models. So, as you move forward, continue experimenting with your own data sets and refining
your own pipeline to unlock the full potential of GenAI in creating smarter, context-aware GenAI systems and models. &gt;&gt; Here's a jaw-dropper to start with. Job
postings for generative AI skills have exploded from just 55 in January 2021 to exploded from just 55 in January 2021 to nearly 10,000 by May 2025. That's a growth of almost 200 times in just 4 years.
Generative AI engineers have gone from niche specialists to some of the most sought-after professionals in tech. In the US, average base salaries now sit around $116,000, with top earners making as much as
$179,000. And that's before bonuses and stock options. So, if you're looking to break into this field, you need more than hype. You need the right skills. In this video, I'll
take you through the top 10 must-know skills to become a generative AI skills to become a generative AI engineer in 2025 and beyond. Each skill is backed with real-world relevance, surprising data, and examples
you can relate to. Let's dive in. First up, at number 10, Python programming. Python is the universal language of AI. It's clean, versatile, and has libraries like NumPy, Pandas, PyTorch, and TensorFlow that
Pandas, PyTorch, and TensorFlow that make building AI systems possible. Imagine trying to train a large language model without Python. It's nearly impossible. Even the world's biggest AI projects, from OpenAI's GPT
to Meta's Llama, are built with Python at their core. Here's the kicker. Almost every AI-related job posting in the US lists Python as a required skill. Not optional, required. That makes it the
single most indispensable tool in your toolkit. At number nine, we have machine learning fundamentals. Generative AI might sound advanced, but at its heart, it's still powered by the basics of machine learning.
This includes supervised and unsupervised learning, decision trees, regression models, and understanding critical concepts like bias, variance, and overfitting. Why is this important? Because without
strong ML fundamentals, you'll struggle to explain why your generative model is producing certain results, or worse, failing. And trust me, in an interview, that's where many candidates lose out. But here's a data twist. The US Bureau
But here's a data twist. The US Bureau of Labor Statistics projects a 32% increase in data science jobs between now and 2033. That's nearly four times faster than the average growth rate across all
occupations. So yes, ML fundamentals are your entry ticket to a booming career. Number eight is deep learning and neural Number eight is deep learning and neural networks. These are the power engines of
generative AI. Without them, we wouldn't have tools like ChatGPT or Stable Diffusion. You need to know how CNNs work for vision tasks, how RNNs process
sequences, and most importantly, how transformers have revolutionized the transformers have revolutionized the entire field. Here's a wild stat. Job entire field. Here's a wild stat. Job postings for AI engineers shot up 143%
in a single year. Companies don't just want someone who knows the terms, they want someone who can explain backpropagation, activation functions, and attention mechanisms with confidence. If you can do that, you're
already ahead of the curve. At number seven, natural language processing, or This is all about teaching machines to understand, process, and generate human language. You'll need to master
language. You'll need to master tokenization, embeddings, and how models like BERT and GPT handle context. Remember, language is messy. Sarcasm, slang, typos, and even cultural context all matter in NLP.
And here's the surprise. The NLP market is projected to reach $43 billion by 2025. Salaries follow the trend. Mid-level NLP Salaries follow the trend. Mid-level NLP engineers can make up to $170,000,
engineers can make up to $170,000, while top experts earn $231,000 So, if you can build systems that actually understand language the way humans do, you won't just be in demand, you'll be in the top salary bracket.
Number six is computer vision and generative models. This is where AI creates images, 3D models, or even video content out of thin air. We're talking about GANs, diffusion models, and autoencoders. The
technologies behind tools like Midjourney and DALL-E. Midjourney and DALL-E. And the demand is massive. Meta pays up to $230,000 for skilled computer vision engineers,
for skilled computer vision engineers, while Tesla averages around $161,000 Why? Because visual AI isn't just for art. It powers autonomous cars, medical imaging, and augmented reality.
In short, if you can make machines see and create, you'll always be in high Halfway down the list, we land on number five. Transformers and large language models. Transformers changed the game. They're
what make ChatGPT, Claude, and Google's Gemini possible. Knowing how LLMs are trained, fine-tuned, and deployed is no longer a nice-to-have. It's mandatory. Here's a stat that proves it. US AI-related job
stat that proves it. US AI-related job listings grew 25% year-over-year in Q1 listings grew 25% year-over-year in Q1 2025. And the median salary for AI roles 2025. And the median salary for AI roles is now close to $157,000.
investing heavily in LLMs and the people who can work with them. At number four, we have prompt engineering. Think of this as the art and science of talking to AI models. Small changes in wording can completely
transform the output of an LLM. This makes prompt engineers incredibly valuable. And here's the jaw-dropper. Salaries for prompt engineers in 2025 now range from $95,000
to $270,000. Just a few years ago, this job didn't even exist. Now it's one of the fastest-growing roles in tech. If you can design prompts that are efficient, reliable, and reusable, you'll stand out
in interviews and at work. Number three is data engineering and ML Ops. is data engineering and ML Ops. Why does this matter? Because even the best AI models are useless without clean, structured data and smooth
deployment pipelines. That's where ML Ops comes in. Handling training workflows, monitoring models, and automating retraining when things go wrong. Here's the surprise fact. AI-related
jobs are twice as likely to offer parental leave and three times more likely to include remote options compared to traditional tech roles. That shows how companies are competing to attract and keep AI talent.
At number two is responsible AI and ethics. This is about making sure your AI isn't biased, unsafe, or harmful. Think of cases where AI misclassifies patients in healthcare, or where models
generate offensive content. These aren't just technical issues, they're legal and reputational risks. By 2025, nearly 47% of large
organizations reported having experienced at least one AI-related issue tied to bias, fairness, or privacy. That's why companies now look for engineers who can ensure their models
are trustworthy and compliant. Being skilled in responsible AI makes you more than just a developer, it makes you a leader. And finally, the number one skill for 2025 is agility. The ability to keep
learning and adapting. Generative AI is evolving faster than any technology in history. What you learn today might be outdated in 6 months. The best engineers
are not just coders, they're constant learners, experimenters, and adapters. Here's the kicker. Generative AI job postings in non-tech roles grew seven
times from 2022 to 2024. While broader IT roles saw a 35 time growth. That's proof that agility is the most valuable skill of all.
Because no matter how the field shifts, adaptable engineers will always thrive. To wrap this up, the median salary for AI engineers in the US is now around
$145,000, with top talent making $200,000 or more. &gt;&gt; We will begin with the agenda for MCP tutorial for beginners. So, firstly, we will have a quick
So, firstly, we will have a quick briefing of what exactly is an MCP, or as it is called, multi-agent collaboration protocol. Followed by that, we will understand some basic protocols and standards in
communication. Next, we will try to relate MCP to the understanding of protocols and standards. Followed by that, we will get started with our demonstration on developing a web scraper agent using MCP.
And for that, we will be needing Node.js, Cursor AI, and Psycho API. So, I will be walking you through those demonstration steps one after the other, and this will be a very beginner-friendly tutorial and practical
demonstration on MCP. Now that I've made myself clear with the agenda, let's get started with MCP. So, what exactly is MCP? MCP? As MCP on their webpage says, MCP is a
Model Context Protocol, or Multi-Agent Collaboration Protocol, is an open-source protocol that standardizes how applications supply context to large language models. Similar to how USB-C unifies hardware connections, MCP offers
a consistent way to integrate AI models with various data sources and tools. Now, let's go to the official documentation of MCP to have a broader documentation of MCP to have a broader overview. So, as they say, MCP is one
such tool. If you remember N8N and all the AI tools and data sources that you used. For example, if you wanted an AI travel agent, as we have an already existing tutorial on our channel, you can go through it for a brief overview.
So, you might be needing an email writing agent, and you might be needing an airport code searching AI agent, hotel searching AI agent, flight booking AI agent, and a lot of agents connected to one another through a complex network
using an agent platform, right? Now, in this particular MCP, that will be minimized. So, you will have a standardized approach where you just eliminate all those boilerplate coding, agents manual search for, databases
manual search for agents which are specifically designed for the purpose. You just explain your initiative, attach You just explain your initiative, attach the API keys, and MCP takes over and
gives you a fully furnished and finished product within no time. That's the main approach of MCP. So, here you can see an overview image where we have the parts MCP host, clients, servers, local data sources, remote services, and local data
local data sources or local knowledge in case if you wanted some open source, then you can also go ahead with that. So, here we have all the quick starts and guides and example and tutorials to
how to begin with that one, and all the explore pages that you want to go ahead into core architecture, the resources available, prompts, tools, transport samplings, and a lot more. And in case if you wanted to contribute your
dedicated resources for the community, you can also go ahead with that. And you can also have some support and feedback for MCP. Now that we have a brief understanding of MCP and still clueless about what exactly is MCP, then let's go
ahead with some standards and protocols so that you can get a template, and you can relate MCP to that template and understand what's the role or importance understand what's the role or importance of MCP in AI agent development. Now,
understanding the standards and protocols. Now, let us imagine that you get a team of people to work on a project of people to work on a project and you are the manager that leads and
directs this particular team for a purpose, right? Now, the complex purpose, right? Now, the complex problem. No one is friend of anyone. Everyone is new to everyone on the team and each and every person in this team
doesn't have a common language in between. For example, person A speaks Mandarin, person B speaks Hindi, person C speaks French, so on. There is no common language between any one of those. Now,
imagine each and every one of these team members as an AI agent or a tool working for the purpose. In this scenario, you might have to hard code the ports of communication, instructions on how to do. You need to go on to each and every
single AI agent and explain the stuff in a very dedicated and a specific way. This could take a very long time. In the same way, if you relate this to a person, then you might have to reach to every single person and speak to them in
their own tongue and explain them the procedures and the steps to be taken to finish the task. Now, how would you do that? It would be a lot complex. Mandarin, you might not even know French or a different language and in that
particular scenario, you might be stuck. What if there was one single language that everyone knew? For example, English. Now, you can get all of them English. Now, you can get all of them together and explain the stuff in one
single call and close it. And what might be so wonderful is that you'll not even waste a lot of time and there is no hard coding, there is no manual effort there, right? In the same way, if you relate this to a web server, then you are the
client, you have the server and you just let's say you're working on Facebook or Instagram, you try to post something, you try to write a command, you try to activity. And these particular activities will go through four
different types of requests. One is the get request, put request, post request, and last is the delete request. Right? Now, if you are connected to a single server, this might not be as tough as possible. But if you are getting
connected to multiple servers, then you might be requiring different protocols every single time. Now, this can be combination of two parts, HTTP and the URL that you're trying to get connected to, right? Now,
to eliminate the efforts between multiple servers and multiple requests at one go, you have one single tool called as API. And the most popular API is the REST API, which has a standard way of
communicating. So, here the REST API works as the language English that we discussed before. In the same way, it has maintained a standard where each and every web client can have his communication to his dedicated server or
multiple servers in one go using one single language or medium or standard single language or medium or standard that is REST API. Without REST API and the standards and protocols, it might be very confusing and you might end up in a
complex situation which might not yield in the type of result you're expecting. Now, if you can relate the development of multiple AI agents for a single of multiple AI agents for a single purpose to the web communication and a
meeting, you also expect if there were a template for development of AI agents which could make the process less complex, just like REST API did in web service, and just like English did in a meeting meeting, then MCP comes into
picture. MCP is your template for it, right? meetings, you have MCP. Earlier, without MCP, we
used to have one tool called NA10, where you had all the tools connected to it, and all the LLMs programmed to it, and you had to do this hard-coded coding for each and every tool that you have included in the network, and each and
every agent that you have related to it, you had to specifically instruct, write down steps, develop a persona to your AI agent, right? There was a lot of stuff. But, if you eliminate this particular complexity with one tool, then that
would be your MCP. You just had to include MCP and tell your purpose, and your job would be done. Now that you have a brief understanding or a general overview of what exactly is MCP, now let's go ahead and build one. Now, the
the project that we are building, which is a web crawler or web scraper, we might need the help of Node.js. So, firstly, we will install and set the environment for Node.js. Followed by that, we will be needing an editor. So,
for that, we will be choosing the Cursor AI editor. And next, we will use an API. So, for that, we will be using the Firecrawl API. And lastly, we can get started with building one once we have all the
required resources. So, let's get started. So, as we discussed before, we'll get started by installing Node.js into our operating system. So, we're working on Windows operating system. So, we might have to go ahead with Windows.
So, I'll choose Windows, and the architecture will be the one which I'm working with. So, that would be X64. But, in case if you are working with uh different operating systems, say Mac OS, then you might have to select ARM or any
suitable architecture that you can be compatible with. For Linux, you'll have And so on. So, since we're working with Windows, so Windows it is and x64 architecture. And click on that Windows installer MSI file, so it will be
getting started to install. And you'll have to click on that Windows downloaded. While it is getting downloaded, let's go to the next web page and download and install the Cursor AI that we will be needing for this
in, create an account and log in. And automatically it'll give you the version we're in Windows. It gave me a Windows operating system. Just sign up or log account, just log in to your account and confirm it's that you and followed by
that, you'll have installed your Cursor AI. Choose your theme. I'll go with the And here you make sure that you have Cursor tab, control key agent, and you can also choose the edition option. I'll go with the privacy mode. In case if you
want the other one, go ahead with that. And uh here you have a few review settings and there you go. Now you're on the new window for Cursor AI. And meanwhile, let's check on the Node.js installation. I think uh the wizard has
already started installing the Node.js setup. So, this is the window. Let's quickly uh go through each and every instance that we have gone through the beginning. So, it started off with some
packages that we would be needing. So, uh Yeah, Chocolatey and uh KB2999. Yeah. Further, we have And the last one is Python 13. Right? Version 3.13.3
is approved. So far, so good. I think we can now get started with the next step. So, here we are on the Cursor AI. So, we might have to get started going through the Cursor AI settings to create a new MCP. So, there are quite a few methods
that you can try out to reach to settings. One is you can go to the file preferences and then you have the cursor settings. You just can use that vertical list or you can go ahead with the search options as well. So, you can use that
where you can just go to the search window and search for the settings. Here you can see MCP on the left window. So, let me also walk you through a few other steps. So, go to the search window and type down Cursor AI or cursor
I think there has been a discrepancy but not to worry, we'll go with the regular approach where we go to the file, preferences, and cursor settings. And there you have the options on the left-hand side. Go to MCP and so far you
is the first time you logged into Cursor AI. So, you just have to add new global MCP server which is available on the blue color button. Just click on that. And now you'll have mcp.json file which is completely empty. You'll have to
write down a few hardcoded codes here which are not too complex related to the earlier ones that you've worked on in A10. So, you might need a few more tools here. So, go to the Fire Crawl website.
Before that, let's go through some awesome awesome MCP servers GitHub link.
related to the MCP. So, let's open a new window and open all these links. What is MCP clients, tutorials, community, legend, server implementations, framework, tips and tricks. Right, for further information you can go through
all these web pages individually. So, back to the home page. Here you have everything you needed. The basic information about exactly is MCP and further you have some server implementations and if you scroll down
so you have awesome MCP service this this where the beginning is and next you have server implementations aggregators art and culture coding execution right customer data platforms so here you have aggregators art and culture dedicated
web spaces so and you have also cloud platforms that you can work along with and moving further you have code executions coding agents command lines as well and communication related stuff so you
can go along and perform your operations customer data or platforms data platforms and much more let's close all these windows and get started with the next stage where we need an API right so for that the
project that we're working on which is web crawler we will be needing an API web crawler we will be needing an API from one of the providers which is Firecrawl API okay that's the one Firecrawl MCP
particular link here to generate an API first you might have to sign up with Firecrawl so before we sign up let's go through beginning so that's the template code that you'll be working on and also here
we have a wide variety of uh uh crawl options media parsing options smart weight actions right reliability first and all those things and also we have the pricing details for now we'll go ahead with the free plan and uh yeah
let me add down my details my email address and you might also have to add a address and you might also have to add a pass so let me type down something that's a good thing to have a really
strong website so that's a sign you need to have strong passwords and also make sure that you don't share your passwords and the API keys that you generate using 5 crawl. Now that my password is a little too longer, I think it should be
Yeah. It's accessible and is a successful account generation. Now you need to check your email and confirm that the everything. So once you are signed up, let's go
through another website for generating our API keys. our API keys. So 5 crawl API, scroll a little bit.
So here you'll find some documentation related to 5 crawl and all the details that you need to go through for API key generation. So 5 crawl and CP server.
looking for. 5 crawl MCP server, that's the one. And here you have overview, content, tools and other resources that you're looking for. Now well this is the template server configuration that you'll be needing. So
make sure that you have an eye on it. And now you will also have the details that you might have to go through. Like basically it'll have the steps that you need to follow in the content page. Click on content.
There you go. So it has all the features and details that you need to go ahead with. And tools you have the crawling, extraction, scraping tools that you can work along with 5 crawl. And few comments here.
Now yeah, the email has not been confirmed yet. Let's try to resend and reconfirm the email so that we just sign up or login and create our API keys as
soon as possible. So there you go. I have verified my email in my phone. So I'll just enter the credentials and sign in. Successfully you are signed in. On the left hand side you can see the overview, playground, extract, activity
logs, usage, API keys, and all the settings. So, here you can see my API key has been by default hidden, so I'll not visualize it now. So, you have two options, either view or copy. So, we will be copying it for the first time to
be on the safe side. So, since we are new to this particular website, so we don't have any activities registered on this particular webpage. So, now we have the API keys. Now, let's go to the content page to
verify the steps that we need to follow. So, we need to install NPM, and we need So, we need to install NPM, and we need to go to the Cursor AI settings and add a new MCP server, which we already did. So, we just need to copy the template
code, and we just need to place it there. And also, don't worry about where exactly to place your API keys in the template. We have that place. So, we are back on the Cursor AI. Now, we will replace this template code with the code
that we just copied. And remember, if you're using Windows, try CMD/C, okay? If you're not on Windows, you can eliminate that CMD/C. But, if you are working in Windows, then you definitely need that CMD/C. Okay,
let's erase this and replace that. So, I'll name my tool as Fire Crawl, and I'll go to the new line, where I'll write down my command. And the command will be the code that I just copied from the Fire Crawl window. So, let me type
down the command in double quotes in new line, and I'll paste the command here, the Fire Crawl window. Control V. is in CMD/C,
if you are working on Windows, that you definitely need it. If you're not omit it. Now, there might be some concerns with the codes. Don't worry. Now, to cross-verify, you can just go back to the Fire Crawl homepage and
cross-verify it with the template for configuration that they have made us available. Let's go back to the fire wall window and at the same time you can see your API key here. So, that's where you need to place your API key.
So, here we are on the fire wall window. We need to copy the API key and place it in the place where they have mentioned your API key. So, if you have cross verified with the configuration code, that's fine. And now
check the double quotes. So, that's the one you need to focus on. Let me kind of if there is a way to zoom it in. Let me try to zoom in.
Yeah, that's the one. CMD/C. If you're on Windows, you definitely need this. If you're not on Windows, you can just choose to omit it. Now, let me just do a one last final check. I think I missed a double quote somewhere.
Yeah, I need to remove it from this place and add it to the end. Okay, remove the double quote in that place. Now, it should be after MCP.
And once everything is done, you can just quickly choose to save and close distance between the API key and the double ampersand places. That's fine. And saving is really simple. Choose those three dots, close, save. And once
to you know, choose to close this particular mcp.json and get back to your agent. have uh your new server that you've just created in the cursor settings. But in a
few times you get no tool connected, no server connected, or there might be some errors. So, in that case you just need to cross verify the NPM installation, Node.js installation, and check all the files that are there or not. But even if
everything is installed and you can see you know, the versions available and if you're still seeing this or watching that particular error, then don't worry. Just close the cursor AI and restart it, or you can also choose the refresh
So, I've closed and restarted it and the same procedure go to file, preferences, cursor settings and there you'll see the MCP models and there you'll find it right there. Correct? Now, close it and go to the window
where you'll find that the server. Yeah, that's the one and here you choose agent instead of anything else. Not money, not ask. I choose the agent
and now I'll write down a simple prompt. Search for Simply Learn on Google. Now, it should give me results which it finds on Google. Maybe it's a homepage, maybe it's a social media handle like LinkedIn or YouTube or Instagram, anything that
Only the term Simply Learn. So, here you can see it has given me three different results. First one is Simply Learn website, second one is Simply Learn YouTube channel and the third one is LinkedIn page. So, let's get started
with the official web page. So, click on the Simply Learn.com. Now, yeah, it's asking me permission open. Now, we're on Simply Learn homepage. Similarly, let's go back and select for
the YouTube page. So, that's the one. Open again, get permissions. So, you're on the Simply Learn YouTube page. Now, I think it might ask for a login for LinkedIn. But anyways, let's try to check if it can
open without a login in to LinkedIn or not. So, yeah, you can see I've logged in to the YouTube. Yeah, it is asking me to login. So, let's quickly login. It might take a couple of seconds.
So, there you go. I've logged in to LinkedIn. Now, open. So, I can see Simply Learn's homepage getting loaded in a second. Yeah, there you go. On LinkedIn, we have opened the Simply Learn's homepage successfully.
&gt;&gt; Imagine you are managing a global supply chain company and where you have to handle orders, shipments and demand forecasting. But unexpected issues arises where sudden shortages like transport delays and the
changes in demand. So, instead of relying on manual adjustments, what if automatically? This AI wouldn't just suggest actions. It would decide, execute, and continuously improve its strategies.
that said, guys, I welcome you all on our today's tutorial on what is Agentic AI. Now, let us start with understanding first the first wave of artificial intelligence, which was predictive
analytics or we could say data analytics and forecasting. What exactly happened like predictive AI focused more on analyzing the historical data, identifying the patterns, and making forecast about the future events. And
these models do not generate any new content, but instead it was predicting outcomes based on the statistical models and machine learning. Now, technically how used to work? So, basically what we had like we used to
take uh Suppose this is the ML model, okay? So, this is taking a structured data, which could be like suppose any past user activity or it could be a transaction record or any sensor reading. For example, you can consider
reading. For example, you can consider say Netflix users watch history, okay? It could be any movie genre, watch time, and the user rating. So, now after this, what we were basically doing is we were doing the feature engineering or
doing the feature engineering or preprocessing, okay? engineering process, we were extracting key features like user watch time trends, preferred genres, and watch frequency. And we could also apply
scaling, normalization, and encoding techniques to basically make data more usable for the ML model. Then, we were using the ML models. Suppose it could be a time series forecasting models like ARIMA, LSTM, and all those given
algorithms, which was basically predicting the future movie preferences based on the historical data. And in the output, guys, Netflix AI recommends new shows or movie based on the similar user patterns.
So, this is how exactly the Netflix model was working incorporating the machine learning model. So, this was exactly the first wave of AI. Now, let exactly the first wave of AI. Now, let us discuss about the second wave of AI.
which was basically content creation and use of conversational AI. So, you know, LM models like ChatGPT became very much popular during the second wave of artificial intelligence. So, what exactly was happening like generative AI
was taking input data and it was producing new content such as text, images, videos, or even code. And these models learn from patterns in large data sets and it was generating human-like outputs.
technology was working. So, basically, first there was a data input. Okay? So, basically, any prompt from the user. So, suppose in the GPT, okay? So, I'll just suppose in the GPT, okay? So, I'll just open GPT all over here and say we are
Suppose we are giving any new prompt. Say such as Say such as write an article Okay? So, this was our given prompt. And after this, what exactly was happening
was tokenization and preprocessing. So, the input text, suppose which I have written all over here, write a article on AI. So, this text was basically split into smaller parts. For example, like uh you could consider certain thing like
this. So, here you have here you have write as one, uh you know, and as next. And similarly, you could carry on, you know, for the other words. Then, what
exactly used to happen that these words were, you know, converted into word embeddings, means the numerical vectors representing words like in a higher dimensional space. And then we used to perform neural network
processing. So, here the LLM processes input such as attention mechanisms, okay, or you know, using these models like GPT-4, BERT, and Llama. And with the help of self-attention layers, they were understanding the context and they
were predicting the next word. Okay. Now, as a result, you are getting output certain thing like this. So, which was basically a generative AI phase. So, this was, guys, our second evolution of AI.
it is basically agentic AI or autonomous AI. Now, what is this, guys? So, the agentic AI actually goes beyond text generation. So, it integrates decision-making, action execution, and autonomous
learning. These AI systems don't just respond to prompts, but they also independently plan, execute, and optimize the processes. you could understand something like this.
this. So, input or receiving any goal. So, user provides any high-level instruction. For
example, it could be like, say, optimize warehouse shipments for maximum It could something be like that. And unlike generative AI, which would generate text, agentic AI executes real-world actions. After this, what
Suppose the prompt that we have given, like optimize warehouse shipments for maximum efficiency, then the next step would have been querying the databases. The AI would pull the real-time data from multiple sources. So, it could be
traditional database like SQL or no SQL, where we are fetching inventory levels or shipment history. Then, it could be a vector database from where it is receiving some unstructured data like past customer complaints and all those
things. Then, with the help of external APIs, it is connecting to like APIs, it is connecting to like forecasting services or fuel price APIs or supplier ERP systems. And these things are like present with this
respect. Then, the third step was the LLM decision making. Now, after acquiring the database, the AI agent processes data through the LLM based reasoning engine. Example like decision rules applied like
it could automate supplier restocking orders. Like if shipment cost is increasing, then it is rerouting shipments through cheaper vendors. And suppose also if weather condition impacts the route, then it is adjusting
the delivery schedules. Now, you could understand how agentic AI is behaving process. Now, next step would be action execution via APIs. So, AI is executing task without human intervention. It is
triggering an API call to reorder stock from a supplier or update the warehouse robot workflows to prioritize fast-moving products or even send emails and notifications to logistic partners and about the changes what is going to
be happening. And after this, finally, it is continuously learning, which is a data flywheel all over here. Okay? The AI is monitoring the effectiveness of its action like it was restocking efficient or did routing shipments, you
know, reduce the cost and all. So, it is mon- monitoring the effectiveness of the action it has taken. And the data flywheel is continuously improving the future decisions. So, basically, it is using reinforcement learning and
fine-tuning to optimize its logic. Okay. Now, let's have a just quick recap about the comparison of the all these three waves of AI. So, basically, predictive AI's main focus was on forecasting the trends, okay? While
generative AI's was creating the content, and agentic AI, on the other hand, which is at the final step right now, is making decision and taking action. So, you could see how the evolution happened of AI in all these
stages. And if you understand about the learning approach, then predictive AI was basically analyzing the historical data. While generative AI was learning from the patterns, like using text, image generation, okay? And but agentic
AI is basically using the reinforcement its learning approach. Now, if you just look at the user involvement in predictive AI, so human is asking for the forecast and all. Here, human is
giving the prompts. But, in the agentic AI, the prompts uh the intervention of human input has become very much minimal. If you could understand the technology, like basically, predictive AI was using
machine learning, time series analytics. So, these kind of, you know, algorithms they were using. Generative AI was using transformers, like GPT, Llama, BERT, and all those things. Now, agentic AI is doing what, guys? It is
using LLM plus APIs plus autonomous execution. So, we have discussed how this workflow is, you know, in a short way, how it is working. And moving ahead, we are also going to discuss through an example how exactly all these
steps, like agentic AI, is working. So, based on the example, you could predictive AI, you know, Netflix recommendation model which they have on their system. And similarly, if we talk about
about ChatGPT, you know, writing articles and all those things. And articles and all those things. And agentic AI, we could imagine like how AI if incorporated in supply chains, how you know things are working out. So
guys, I hope so you would have got a brief idea regarding the three waves of AI. Now, let us move ahead and a bit understand about what is exact agentic AI. Now guys, let us understand the
difference between generative AI and agentic AI. So, let us first, you know, deep dive into what exactly is a generative AI, okay? So, as you can see all over here that generative AI models generally are taking input query, okay?
And they are processing it using LLM or large language model, and basically returning a static response without taking any further action. So, in this taking any further action. So, in this case, for example, a chatbot like
uh ChatGPT, you know, it is taking the input from the user. So, as I've shown you earlier that uh say, suppose I've given an input like write a blog post on AI in healthcare. So, when I have written this given you know, user input
or given the query, so when it goes to the large language model, this model is actually, you know, tokenizing all these input query, and it is retrieving the relevant knowledge from its training data. And it generate
text based on the patterns. Now, we give the prompt, then LLM processes it, okay? And then we are getting the given output. So, now this is basically how,
you know, generative AI is working. So, you could see all over here, we have GPT model, we have DALL-E, we have Codex. So, these are some of the, you know, amazing, you know, generative AI models, okay?
Now, let us discuss a bit about DALL-E, which is actually a, you know, realistic image generation, you know, GenAI. So, like DALL-E is described as, you know, the realistic image generation model by the OpenAI. And this actually is a part
of, you know, generative AI category alongside with GPT, which is basically for human-like language creation purposes. This model was created. And you could have also Codex for like it could be used for advanced code
generation purposes. Now, let us discuss a bit about DALL-E. So, DALL-E is like a deep learning model basically, which is designed to generate realistic images from the text prompt. And it can create highly detailed and creative visuals
users. So, some of the aspects of DALL-E like you could have all over here like text to image generation where users can input text prompts and DALL-E can generate unique images based on those
description. The images generated by DALL-E are highly realistic and creative, okay? And it can generate photorealistic images, artistic illustration, and even surreal or imaginative visuals. They would also
have customization and variability where it is allowing variation of an image edits based on text instruction and multiple style. So, this is also part of a generative AI model and it is This tool is actually
playing a very amazing role. So, I will show you one example like how generative AI is actually working in the image generation purposes. So guys, as you can see all over here, I have opened this generative AI tool called DALL-E. Let us
give a prompt to DALL-E and let us see how the image is generated. So, let's say we want a futuristic city
filled with neon skyscraper and holographic billboards.
robots. And we can have people wearing let's just say high-tech, you know, let's include some technology.
uh DALL-E is trying to create a image. So, this is how actually generative AI is working. So, output comes up. Now, you could see all over here
that uh this image which is generated, basically, this is generated by AI and you could see based on our prompt, it has given like the kind of, you know, based on this. Now, so this is one of the amazing
this, guys. Okay? Now, guys, let us discuss about agentic AI or autonomous decision-making and action execution. So, you could see this diagram all over here. So, agentic AI like unlike the
generative AI, it is not generating responses, but it is also executing a task autonomously based on the given query. For example, like if you take AI in managing a warehouse inventory, okay? Suppose we want to optimize the
warehouse shipment for the next quarter. So, here what is going to happen? So, first the agent is going to receive its goal all over here, okay? And this AI agent, you know, is going to query the external
data sources. So, it could you know, for example, it could be your you know, inventory databases or logistics API. And then it retrieves real-time inventory levels and it demands the given forecast. Okay. Now,
at here it is going to make the autonomous discussions and the kind of output we are going to get will be kept in observation by this agent, okay? So, basically, it is going to analyze the current warehouse stock,
predict demand for the next quarter, check the suppliers availability, and automate the restocking if inventory is below the given threshold. So, uh for example, you could imagine uh you know, suppose based on the you know,
output what we are going to get all over here. So, based on this output, we could here. So, based on this output, we could get certain thing like this. Like say, current inventory level
inventory level like say, 75% percent capacity. it could have also other thing like say,
say, demand forecast say, 30%
expected in uh quarter two. And also, it is going to go say, like And also, it is going to go say, like say, reordering
initiated. So, this is the output what we are going to get based on the supply chain management in example what we are trying to get. So, as we have seen in generative AI, user is giving the input,
okay, prompt, then it is using LLM model to generate the given output. But, agentic AI is doing what, guys? It is going It is going to take an action, you know, beyond just generating a text. So, in this scenario, it is querying the
inventory databases, it is automating the purchase order, it is going to select the optimal shipping providers which could be, you know, suitable for the given company. It is going to continuously refine the strategies based
on the real-time feedback. So, guys, let's recap once more. So, if you talk about the function base, then GNA is more concerned with producing a written content or a visual content, okay? And even it can code from the pre-existing
guys, decision-making, taking actions towards a specific goal, and it is focused on achieving the objectives by interacting with the environment and making the
autonomous decision. GenAI is exactly relying on the existing data to predict and generate content based on say patterns it has learned during its training phase. But it does not adapt or evolve from its experiences. Whereas if
I talk about agentic AI, it is adaptive. So it is learning from its actions and experiences. It is improving over time by analyzing the feedback, adjusting its behavior to meet objectives more effectively. With the help of GenAI,
human input is essential to the prompt so that, you know, basically with the help of that it could go into the LLM model and it could generate the given, you know, output based on your prompt. Once
you set up the agentic AI, it requires like minimum human involvement. It operates autonomously, making decisions and adapting to changes like without continuous human guidance. And it can even learn in real time. So that's with
the beauty of agentic AIs. So we have given one example of GenAI like basically giving prompt to the ChatGPT or DALL-E, okay? And agentic AI one example could be your supply chain management system. Now, let us bit deep
dive into understanding the technical aspects of how agentic AI is exactly working. Now, guys, let us try to understand how agentic AI is exactly working. So there is actually a four-step process of, you know, how
four-step process of, you know, how agentic AI exactly works. So the first step is, you know, perceiving. Where basically what we are doing is we are gathering and processing information from databases, sensors, and digital
environments. And also, the next step is reasoning. So, with the help of large language model as a decision-making engine, it is generating the solutions. If we talk about the third step, which is acting, so it is integrating with
external tools and softwares to autonomously execute the given task. And finally, it is learning continuously to improve through the feedback loop, which is also known as the data flywheel. Okay. Now, let us explore each of the
step one by one and let us try to understand. So, if you talk about perceiving, okay? So, this is actually the first step where agentic AI is actually stepping up. So, it is doing the perception, where what exactly is
happening, guys? That AI is collecting data from multiple sources. So, this data could be from database, okay? Like your traditional and vector databases, okay? So, it could be graph QL like vector database means the same. And if
we talk about other from data, it could be from APIs like it is fetching real-time information from external systems. It is basically taking data from the IoT sensors like for real-world applications
like robotics and logistics. And also, it could take, you know, data from the user inputs also. Like it could be text command, voice commands, or a chatbot interaction. Now, how it is exactly working, guys? So, basically, let us
recollect everything technically and let us see how this is happening. So, the first step which is going in perceiving is the data extraction, where exactly the AI agent queries the structured databases like SQL or NoSQL
for relevant records. It is also using vector databases to retrieve any semantic data for context-aware responses like it could anything, you know, it is trying to find out.
Okay. So, next, after it has got the data extraction, it goes for feature extraction and pre-processing where AI is filtering the relevant features from the raw data. For example, like a fraud
detection AI is scanning the transaction log for anomalies. The third thing it is entity recognition and object detection. So, AI uses basically computer vision to detect objects and images and then it is applying the named entity
recognition. This is a technique, okay? Uh to extract the critical terms from the given text also. So, we have three uh step-by-step process which is happening in uh perceiving. The first one is data extraction, second one is a
feature extraction and pre-processing, the third one is like entity recognition and object detection. So, uh let us take a very simple example like uh AI-based customer support system. So, if we consider an agentic AI assistance like
for a customer service. So, say a customer is asking, "Where is my order?" So, the AI queries multiple databases all over here. Suppose, it is going to query the e-commerce order database to retrieve the order status or it could go
to the logistics API to track the real-time shipment location. Also, it history to provide personalized response. The result what we get all over here is that the AI is fetching the tracking details, identifying any delays
if it is happening, and suggesting the best course of action. Now, uh the next best course of action. Now, uh the next step is reasoning, okay? Now, AI's understanding and decision-making and problem-solving is making agentic AI
greater. So, here what is exactly happening like once the AI has perceived the data, now it should start reasoning it, okay? So, the LM model acts as a orchestrating AI processes and integrating with a specialized models
for various function. So, if you talk about the key components like here used in the reasoning, it could be LLM based decision-making. So, AI agents could use models like LLMs, like GPT-4, Cloud, Llama to interpret a user intent and
generate a response. It is basically coordinating with smaller AI models for domain-specific tasks. Like it could be like financial prediction or medical diagnostics. So, these could be you know given example. Then it is using
a retrieval augmented generation or RAG model, okay? To with the help of which AI is enhancing the accuracy, you know, by retrieving any proprietary data from the company's databases. For example, like instead of relying on GPT-4's
knowledge, the AI can fetch company-specific policies to generate the accurate answers. So, this could be the one. And in in the reasoning, the final step is AI workflow and planning. So, this is a multi-step reasoning where
AI is breaking down complex task into logical step. For example, like if asked to automate a financial report, AI is retrieving the transaction data, analyzing the trend, and it is formatting the result. Also, for
example, you could use this in supply chain management. So, suppose consider there is a logistics company which is using the agentic AI to optimize what could be the you know shipping routes, you know. So, a supply
chain manager requesting the AI agent to find the best shipping route to reduce the delivery cost. So, the AI processes real-time fuel prices, traffic conditions, and weather report. So, using LLM plus data retrieval, it finds
out the optimized routes and selects the cheapest carrier. Result you get is that AI chooses the best delivery option. So, here the cost is reduced and improving efficiency. So, this is one of the use cases, guys.
So, after perceiving you get is reasoning. Okay. Now, let us move ahead and discuss about the third step, which is act. So, in this step, basically, what is happening like AI is taking autonomous actions.
So, unlike generative AI, which stops at generating content, so agentic AI takes the real-world action, okay? How AI is executing tasks autonomously, guys? So, basically, first step is like here the integration with APIs and software could
be happen, where AI can send automated API calls to the business systems. For example, like reordering the stock from the supplier's API. So, suppose any inventory level is going down, so it could, you know, reorder that particular
stock from the supplier's API. So, it is interacting with the given API. Now, it could also automate the workflows like AI executes multi-step workflows without human supervision. So, here like AI can handle like insurance claims by
verifying the documents, checking policies, and approving the payouts. And finally, AI could operate within predefined business rules, okay? To prevent any unauthorized actions, also. So, ethical AI is a basically being
worked in this direction. For example, like AI can automatically process claims 10,000 US dollars, you know, but it is requiring the human approval for the higher amounts. So, based on, you know,
insurance and policy-making stuff, so agentic AI could be, you know, really helpful in this scenario. One example, like let us consider. So, let's say we have this agentic AI managing an IT support system. So,
suppose a user says, "My email server is down." So, the AI can diagnose the issue, restart the server, and confirms the given resolution. Now, if it is unresolved, then AI escalates to a human technician. Then, it results into, you
know, AI is fixing the issues autonomously, reducing the downtime. Okay. So, this is where your action or act is coming up into the picture. Now, if you go on to the next and final step, which is learning. So, learning
basically, with the help of data flywheel, it is continuously learning. Okay? So, this is the feedback loop all over here, which is the data flywheel. So, how AI learns over the time? If we ask this question, so what is exactly
happening that it is interacting with the data collection. Suppose, AI logs successful and failed actions. For example, like if users correct AI-generated responses, the AI is learning from those corrections. Second
thing what you could do is you could model you could fine-tune the model and do reinforcement learning. So, AI adjusts its decision-making models, you know, basically to improve future accuracy. It uses reinforcement learning
basically to optimize workflows based on past performance. Okay? Now, third step could be automated data labeling and self-correction. So, here what is happening that AI is labeling and categorizing past interactions to refine
its knowledge base. Example, like AI autonomously is updating frequently asked answers based on the recurring user queries. So, in this way AI is learning over the time. Example one you could consider so say we
have this AI is optimizing any financial fraud detection. So, say this is consider that this is a bank which is AI-powered which has this AI-powered fraud detection system. So, AI is
analyzing these financial transaction and it is detecting any suspicious activity. And if flagged, the transactions are false AI is learning to reduce these false alerts. So, over the time AI is
improving the fraud detection accuracy like minimizing disruptions for the customer. So, in this way AI is getting smarter over the time like reducing the false alerts and also the financial fraud. So, let's have a just quick recap
of what we studied right now. So, agentic AI works in four steps. The first step is perceiving where AI is gathering data from databases, sensors, and APIs. The next step is reasoning.
So, it is using LLM to interpret task, applies logic, and generating the solution. The third step is acting. So, here AI is integrating with external systems and automating the task. And finally, it is learning. So, AI is
improving over the time, you know, via feedback loop or which is basically called as data flywheel. So guys, now let us see this diagram and try to understand what this diagram is trying to say. So, the first thing you could
see an AI agent all over here. So, this is an AI agent which is basically an autonomous system. So, which has the capability of perceiving its environment, making decision, and executing actions without any human
intervention. Now, AI agent is acting as the central intelligence, okay, in this given diagram. And it interacts with the user, okay, and various other data sources. It processes input, queries databases,
makes decision using a large language model. And it is executing action, and it is learning from the given feedback. Now, the next step you could see the LLM model. So, if we talk about LLMs, these are the large language model which is
kind of an advanced AI model trained on massive amount of text data to understand, generate, and reason over natural language. Now, if I talk about this LLM, so this is actually acting as the reasoning engine all over here. And
it is interpreting the user inputs and making informed decision. It is also retrieving relevant data from the databases, generating responses. It can also coordinate with multiple AI models for different task. Like it could be
content generation, okay, predictions, or decision-making. Now, when the user is asking a chatbot, like for example, let's say, what is my account balance? So the LLM processes the query, retrieves the relevant data, and
responds the given bank balance accordingly. But now if you look at the kind of database the LLM is interacting, so we have the traditional database and the vector database. So here, if I say
the database like AI agent basically is querying the structured database. So suppose structured database like it could be a customer records or inventory data or it could be any transactional log also. So traditional databases
basically store well-defined in a structured information. Okay. So for example, like when a bank AI assistant is processing a query like show my last five transaction, so it is basically fetching the information from a
traditional SQL-based database. Next, we have this vector database also, guys. So vector database is a specialized kind of a database for storing unstructured embeddings, images, or audio representations. So guys, like unlike
traditional databases that store exact values, vector databases store in a high-dimensional mathematical space. It allows AI models to search semantically similar data instead of like exact matches. Now AI is retrieving the
contextual information from the vector databases, which is actually enhancing the decision making. It is improving the AI memory by allowing the system also to search for, you know, conceptually similar past
interaction. Let us take an example to understand this. For example, we have discussed about a customer support chatbot. So suppose if it queries a vector database to find out similar past tickets like when responding to a
engine could use a vector database to find out similar products on a user's past preferences. So this could be done in that scenario.
Also, some of the like popular vector databases could be like Facebook's AI databases could be like Facebook's AI similarity search, Pinecone or Weaviate. These are the certain uh amazing uh vector databases. Then you could see the
next step is, you know, after it has worked on these given data, it is performing the action. So, the action component is referring where AI's agent has this ability to now execute task autonomously after the reasoning is
done. So, AI is integrating with external tools, APIs, or automation software to complete the given task. It does not provide only information, but it is actually uh say, you know, performing the given action. So, for
example, like in a customer support, the AI can automatically reset a user's password after verifying the identity. If you talk about in finance, then AI can approve a loan also, like based on the predefined eligibility criteria.
Now, finally, we have the data flywheel. So, data flywheel is a continuous feedback loop where AI is learning from the past interactions, refining its models, and it is always improving over the time. Now, every time like the AI is
interacting with the data or taking an action or receiving a feedback, that information is fed into this model. So, this is creating a self-improving AI system that is becoming smarter over the time. So, the data flywheel is
allowing AI to learn from every interaction, and AI is becoming more efficient by continuously optimizing responses and refining strategies. Best thing in could be used in a fraud detection. So, in this, the AI is going
to learn from the past fraud cases, and it is going to detect new fraudulent patterns and more effectively. Chatbots also can learn from user feedback and improve the responses. And finally, you have the model customization, which is
basically you are trying to fine-tune the AI models on specific business need or any industry requirement. So, AI models are not static, like they can be adapted and optimized for a specific task. So, custom fine-tuning is
actually improving the accuracy and domain-specific application, like it cybersecurity. So, a financial institution, uh say, fine-tuning an LLM to generate an investment advice, okay,
on a historical market trends. That could be one use case. Or in healthcare, if we discuss, like uh the healthcare provider is fine-tuning then AI model to interpret the medical reports and recommend the treatments. So, guys,
based on the given diagram, you would have got a brief idea like how, uh you have got a brief idea like how, uh you know, agentic AI is working. Now, if we discuss about the future of agentic AI, then, guys, I would say it looks very
much promising because it is keep improving itself and it is finding new ways to be useful. Like with better machine learning algorithms and smarter decision-making, these AI system will be more uh independent handling complex
tasks on their own. And believe me, in industries like healthcare, finance, customer service, they have already started to see how AI agents can make more impact and it could be more efficient from personalization
perspective, you know, managing resources, and many more other things. So, as the system continue to learn and adapt, I think so they will be opening up even more possibilities, helping businesses grow, improving how we live
businesses grow, improving how we live and work. Now, I would say that uh in conclusion that agentic AI is actually paving the way for new opportunities, which was assisting with generating content or predicting the data, you
know, uh responding to any queries, but agentic AI can perform techniques independently with minimal human effort. And agentic AI has become self-reliant in decision-making way, and it is making a very big differences in industry like
healthcare, logistics, customer services, which is enabling companies to be more efficient. As a result, it is providing better services to their providing better services to their clients.
machine learning, so let's see if we can give a little bit more of a concrete definition. What is machine learning? Machine learning is a science of making computers learn and act like humans by feeding data and information without
being explicitly programmed. We see here we have a nice little diagram where we have our ordinary system, and your computer nowadays you can even run a lot phones have advanced so much. And then
with artificial intelligence and machine learning, it now takes the data and it learns from what happened before, and then it predicts what's going to come next, and then really the biggest part right now in machine learning that's
going on is it improves on that. How do we find a new solution? So, we go from stuff and understanding how it fits together to predicting what it's going to do to post scripting, coming up with a new solution. And when we're working
on machine learning, there's a number of different diagrams that people have posted for what steps to go through. A lot of it might be very domain specific. So, if you're working on photo identification versus language versus
medical or physics, some of these are switched around a little bit or new to the domain. This is kind of a very general diagram. First, you want to know what it is you're wanting to predict. Then you're going to be
defined an objective, you need to collect the data that matches. You spend a lot of time in data science collecting data. And the next step, preparing the data is clean going in. There's the old saying, bad data in, bad answer out or
bad data out. And then once you've gone through and we've cleaned all this stuff coming in, then you're going to select the algorithm. Which algorithm are you algorithm. In this case, I think we're going to be working with SVM, the
support vector machine. Then you have to test the model. Does this model work? Is this a valid model for what we're doing? And then once you've tested it, then you want to run your prediction. You want to run your prediction or your choice or
whatever output is going to come up with. And then once everything is set you want to go ahead and deploy the model. And remember I said domain specific, this is very general as far as the scope of doing something. A lot of
models you get halfway through and you realize that your data is missing new data because you've run a test in here someplace along the line. You're answers I need." So, there's a lot of things that are domain specific that
become part of this model. This is a very general model, but it's a very good model to start with. And we do have some basic divisions of what machine learning instance, do you want to predict a category? Well, if you're categorizing
thing, that's classification. For instance, whether the stock prices will increase or decrease. So, in other words, I'm looking for a yes-no answer. Is it going up or is it going down? And in that case, we'd actually say, is it
going up, true? If it's not going up, it's false, meaning it's going down. This way, it's a yes-no, 0-1. Do you want to predict a quantity? That's regression. So, remember we just did classification, now we're looking at
regression. These are the two major divisions in what data is doing. For instance, predicting the age of a person based on the height, weight, health, and different factors, you might guess how old a person is. And then there are a
lot of domain-specific things. Like, do you want to detect an anomaly? That's anomaly detection. This is actually very popular right now. For withdrawal anomalies. You want to know when someone's making a withdrawal that
might not be their own account. We've actually brought this up because this is predicting the stock, whether to buy stock or not, you want to be able to market is an anomaly, use a different
is going on, you got to pull out new information in there, or is this just the norm? I'm going to get my normal return on my money invested. So, being able to detect anomalies is very big in data science these days. Another
question that comes up, which is on what we call untrained data, is do you want to discover structure in unexplored data? And that's called clustering. For instance, finding groups of customers with similar behavior given a large
database of customer data containing their demographics and past buying records. And in this case, we might notice that anybody who's wearing certain stores. Or whatever it is, they're going to make certain purchases.
By having that information, it helps us to market or group people together so that we can now explore that group and find out what it is we want to market to And that might also work in just about any arena. You might want to group
people together whether they're based on their different areas and investments and financial background whether you're going to give them a loan or not before you even start looking at whether they're valid customer for the bank, you
different areas and group them together based on unknown data. So you know you tell you but you want to cluster people together that come together. Let's take a quick detour for quiz time. Oh, my favorite. So we're going to have
a couple questions here under our quiz time and we'll be posting the answers in these part two of this tutorial. So let's go ahead and take a look at these you'll get them all right. It'll get you thinking about how to process data and
what's going on. Can you tell what's happening in the following cases? Of cup of coffee and you have your check box and your pen trying to figure out what's your next step in your data science analysis. So the first one is
grouping documents into different categories based on the topic and categories based on the topic and content of each document. Very big these days. You know you have legal documents, you have maybe it's a sports group
documents, maybe we're analyzing newspaper postings. But certainly having that automated is a huge thing in today's world. B, identifying handwritten digits in images correctly. So we want to know whether
they're writing an A or capital A, B, C. What are they writing out in their hand digit of their handwriting. C, behavior of a website indicating that the site is not working as designed. D, predicting salary of an individual based on his or
her years of experience. You know HR hiring set up there. So stay tuned for these questions when we get to the part two of this tutorial, or you can just simply write at the bottom and send a note to Simply Learn and they'll follow
up with you on it. Back to our regular content. Now, these last few bring us into the next topic, which is another way of dividing our types of machine learning, and that is with supervised, unsupervised,
and reinforcement learning. Supervised learning is a method used to enable machines to classify and predict objects, problems, or situations based on label data fed to the machine. And in here, you see we have a jumble of data
with circles, triangles, and squares, and we label them. We have what's a circle, what's a triangle, what's a square, and we have our model training, answer. Very important when you're doing supervised learning, you already know
the answer to a lot of your information coming in. So, you have a huge group of data coming in, and then you have new data coming in. So, we've trained our model, the model now knows the difference between a circle, a square, a
triangle. And now that we've trained it, we can send in, in this case, a square that the top one's a square, and the next one's a circle. And you could see that this is uh being able to predict whether someone's going to default on a
earlier. Supervised learning on stock market, whether you're going to make money or not. That's always important. And uh if you are looking to make a fortune in the stock market, keep in mind, it is very difficult to get all
the data correct on the stock market. It is very uh it fluctuates in ways you really hard to predict. So, it's quite a roller coaster ride if you're running you start realizing you really have to dig for new data. So, we have supervised
learning, and if we have supervised, we should need unsupervised learning. In unsupervised learning, machine learning model finds the hidden pattern in an unlabeled data. So, in this case, instead of telling it what the circle
is, and what a triangle is, and what a square is. It goes in there, looks at groups them together. Maybe it'll group it by the number of corners. And it notices that a number of them all have three corners, a number of them all have
four corners, and a number of them all have no corners. And it's able to filter We talked about that earlier with looking at a group of people who are out shopping. We want to group them together to find out what they have in common.
And of course, once you understand what people have in common, maybe you have store, or you have five of them are customer at your store, and they have a lot in common with five others who are not customers at your store. How do you
customers at your store yet? They fit the demographics of who's going to shop your store, not the one next door. Of course, this is a simplified version. between a triangle and a circle, which
is might not be so easy in marketing. Reinforcement learning. Reinforcement learning where an agent learns how to behave in an environment by performing actions and seeing the result. We have here where the in this case, a baby.
infant for this slide, cuz the reinforcement learning is very much in its infant stages. But it's also probably the biggest machine learning future. It's going to be coming up over the next few years is reinforcement
learning and how to make that work for us. And you can see here where we have our action. And the action in this one, it goes into the fire. Hopefully the baby didn't It was just a candle, not a giant fire pit like it looks like here.
And the baby comes out, and the new state is the baby is sad and crying, cuz they got burned on the fire. And then maybe they take another action. The one taking the actions. And in this case, they didn't go into the fire. They
baby's happy and laughing and playing. Reinforcement learning is very easy to understand, cuz that's how as humans, that's one of the ways we learn. We learn whether it is, you know, you burn yourself on the stove, don't do that
anymore. Don't touch the stove. In the big picture, being able to have machine learning program on or AI be able to do this is huge cuz now we're starting to learn how to learn. That's a big jump in the world of computer and machine
learning. And we're going to go back and just kind of go back over supervised versus unsupervised learning. Understanding this is huge because this is going to come up in any project you're working on. We have in supervised
learning we have labeled data, we have direct feedback. So someone's already gone in there and said, yes, that's a triangle. No, that's not a triangle. And then you predict an outcome. So you have a nice prediction. This is this this new
what it's going to be. And then with unsupervised training, it's not labeled. So we really don't know what it is. There's no feedback. So we're not we're not telling it whether it's a triangle or a square, we're not telling
it to go left or right. All we do is we're finding hidden structure in the data. Grouping the data together to find out what connects to each other. And then you can use these together. So imagine you have an image and you're not
sure what you're looking for, so you go in and you have the unstructured data, find all these things that are connected together. And then somebody looks at those and labels them. Now you can take that labeled data and program something
to predict what's in the picture. So you can see how they go back and forth and different tools together to make a bigger picture. There are many interesting machine learning algorithms. Let's have a look at a few of them.
of what's out there. And these are some of the most important ones that are currently being used. We'll take a look at linear regression, decision tree, and the support vector machine. Let's start with a closer look at linear regression.
Linear regression is perhaps one of the most well-known and well-understood algorithms in statistics and machine learning. Linear regression is a linear model. For example, a model that assumes a linear relationship between the input
variables X and the single output variable Y. And you'll see this if you remember from your algebra classes, Y = MX + C. Imagine we are predicting distance traveled Y from speed X. Our linear regression model representation
linear regression model representation for this problem would be Y = M * X + C for this problem would be Y = M * X + C or distance = M * speed + C. Where M is the coefficient and C is the Y intercept. And we're going to look at
two different variations of this. First, we're going to start with time is constant. And you can see we have a bicyclist. He's got a safety gear on, thank goodness. Speed equals 10 m/s. And so over a certain amount of time, his
distance equals 36 km. We have a second bicyclist who's going twice the speed or bicyclist who's going twice the speed or 20 m/s. And you can guess if he's going then he's going to go twice the distance. And that's easy to compute. 36
distance. And that's easy to compute. 36 * 2, you get 72 km. And so if you had the question of how fast would somebody who's going three times that speed or 30 m/s is, you can easily compute the distance. In our head, we can do that
without needing a computer. But we want to do this for more complicated data. So But let's just take a look at that and what that looks like in a graph. So in a linear regression model, we have our distance to the speed and we have our M
equals the V E slope of the line. And we'll notice that the line has a plus slope. And as speed increases, distance also increases. Hence, the variables have a positive relationship. And so
your speed of the person which equals Y = MX + C, distance traveled in a fixed interval of time. And we could very easily compute either following the line or just knowing it's three times 10 m/s that this is roughly 102 km distance
that this third bicyclist has traveled. One of the key definitions on here is positive relationship. So, the slope of the line is positive, as distance increase, so does speed increase. Let's take a look at our second example where
we put distance as a constant. So, we have speed equals 10 m per second. They have a certain distance to go and it takes him 100 seconds to travel that bicyclist who's still doing 20 m per second. Since he's going twice the
speed, we can guess he'll cover the distance half the time, 50 seconds. And of course, you could probably guess on the third one, 100 divided by 30, since he's going three times the speed, you can easily guess that this is 33.333
seconds time. We put that into a linear regression model or a graph. If the let's see the relationship between speed and time. And as time goes up, the amount of speed to go that same distance goes down. So, now your m equals a minus
v e slope of the line. As the speed increases, time decreases, hence the variable has a negative relationship. Again, there's our definition, positive relationship and negative relationship, dependent on the slope of the line. And
with a simple formula like this, um and even a significant amount of data, let's implementation of linear regression. And have this data set where we have x y, x
equals 1 2 3 4 5, standard series, and the y value is 3 2 2 4 3. When we take points on a graph, you can see there's
probably eyeball a line through the middle of it. But we're going to calculate that exact line for linear regression. And the first thing we do is we come up here, we have the mean of x i. And remember, mean is basically the
i. And remember, mean is basically the average. So, we added 5 + 4 + 3 + 2 + 1 and divide by 5. And that simply comes out as 3. And then we'll do the same for Y. We'll go ahead and add up all those numbers and divide by five and we end up
with a mean value of Y of I equals 2.8. Where the XI references is an average or means value and the YI also equals a means value of Y. And when we plot that, you'll see that we can put in the Y equals 2.8 and the X equals 3 in there
little different color so you can sort it out with the dash lines on it. And the linear regression, the linear regression model should go through that dot. Now, let's find our regression equation to find the best fit line. I
equals MX plus C. So, we're looking for M and C. So, to find this equation for our data, we need to find our slope of M and our coefficient of C. And we have Y
and our coefficient of C. And we have Y equals MX plus C, where M equals the sum of X minus X average times Y minus Y average or Y means and X means over the sum of X minus X means squared. That's how we get the slope of the value of the
line. And we can easily do that by creating some columns here where we have XY. Computers are really good about iterating through data and so we can easily compute this and fill in a graph of data. And in our graph, you can
easily see that if we have our X value of one and if you remember the XI or the means value was three, 1 minus 3 equals a negative two and 2 minus 3 equals a
negative one, so on and so forth. And we can easily fill in the column of X minus can easily fill in the column of X minus XI, Y minus YI, and then from those we can compute X minus XI squared and X minus XI times Y minus YI. And you can
ahead and sum the different columns for the answers we need. So, we get a total of 10 for our X minus XI squared and a total of two for X minus XI times Y minus YI. And we plug those in, we get 2/10, which equals 0.2. So, now we know
the slope of our line equals 0.2. So, we can calculate the value of C. That'd be the next step is we need to know where crosses the Y axis. And if you remember I mentioned earlier that the linear regression line has to pass through the
means value, the one that we showed earlier. We can just flip back up there to that graph and you can see right here, there's our means value, which is here, there's our means value, which is three x = 3 and y = 2.8. And since we
know that value, we can simply plug that into our formula y = 0.2x + c. So, we into our formula y = 0.2x + c. So, we plug that in and we get 2.8 = 0.2 * 3 + c and you can just solve for c. So, now we know that our coefficient = 2.2. And
once we have all that, we can go ahead and plot our regression line y = 0.2 * x + 2.2. And then from this equation, we can compute new values. So, let's
predict the values of y using x = 1 2 3 4 5 and plot the points. Remember the 1 2 3 4 5 was our original x values. So, now we're going to see what y thinks they are, not what they actually are. And when we plug those in, we get y of
designated with y of p, you can see that x = 1 = 2.4, x = 2 = 2.6 and so on and so on. So, we have our y predicted values of what we think it's going to be
we plot the predicted values along with the actual values, we can see the things that's very important with linear regression and any of these models is to calculate the error on all of our different values and you can see over
different values and you can see over here we plotted x and y and y predict. of see what the error looks like there between the different points. So, our goal is to reduce this error. We want to minimize that error value on our linear
regression model. Minimizing the distance. There are lots of ways to minimize the distance between the line and the data points, like sum of squared errors, sum of absolute errors, root mean square error, etc. We keep moving
this line through the data points to make sure the best fit line has the least square distance between the data points and the regression line. So, to recap with the very simple linear regression model, we first figure out
middle and then we slowly adjust the line to minimize the error. Keep in mind this is a very simple formula. The math gets even though the math is very much the same, it gets much more complex as we add in different dimensions. So, this
we add in different dimensions. So, this is only two dimensions, y = mx + c, but is only two dimensions, y = mx + c, but you can take that out to x, z, i, j, q, they can plot a linear regression model on all of those using the different
formulas to minimize the error. Let's go ahead and take a look at decision trees, a very different way to solve problems than the linear regression model. Decision tree is a tree-shaped algorithm used to determine
a course of action. Each branch of a tree represents a possible decision, occurrence, or reaction. We have data which tells us if it is a good day to play golf. And if we were to open this data up in a general spreadsheet, you
can see we have the outlook, whether it's a rainy, overcast, sunny, it's a rainy, overcast, sunny, temperature, hot, mild, cool, humidity, windy, and did I like to play golf that day, yes or no. So, we're taking a
census and certainly I wouldn't want a computer telling me when I should go play golf or not, but you could imagine if you got up in the night before you're and says tomorrow would be a good day for golf for you in the morning and not
a good day in the afternoon or something like that. It becomes very beneficial coming out now where it gives you suggestions and lets you know what would be fit the match for you for the next day or the next purchase or the next uh
whatever, you know, next meal out. In this case, is tomorrow a good day for playing golf based on the weather coming in. And so, we come up and let's determine if you should play golf when the day is sunny and windy. So, we found
sunny and windy. And suppose we draw our tree like this. We're going to have our humidity, and then we have our normal, which is if it's if you have a normal humidity, you're going to go play golf.
we look at the outlook. And if the outlook is sunny, overcast, or rainy, it's going to change what you choose to do. So, if you know that it's a very high humidity and it's sunny, you're probably not going to play golf cuz
fighting off the mosquitoes that are out joining you to play golf with you. Maybe to play in the rain. But if it's slightly overcast and you get just the right shadow, that's a good day to play golf and be outside out on the green.
Now, in this example, you can probably make your own tree pretty easily cuz But the question is, how do you know data? What if this is much more complicated data where it's not
something that you would particularly understand, like studying cancer. They take about 36 measurements of the cancerous cells, and then each one of those measurements represents somehow bulbous it is, how extended it is, how
sharp the edges are. Something that as a human we would have no understanding of. So, how do we decide how to split that data up? And is that the right decision going to come up. Is this the right decision tree? For that, we should
decision tree? For that, we should calculate entropy and information gain. Two important vocabulary words there are the entropy and the information gain. Entropy. Entropy is a measure of randomness or impurity in the data set.
Entropy should be low. So, we want the chaos to be as low as possible. We don't want to look at it and be confused by the images or what's going on there with mixed data. And the information gain, it is a measure of decrease in entropy
after the data set is split, also known as entropy reduction. Information gain information that we get out of the split to be as high as possible. Let's take a look at entropy from the mathematical side. In this case, we're going to
side. In this case, we're going to denote entropy as I of P of N N, where P is the probability that you're going to play a game of golf, and N is the play the game of golf. Now, you don't really have to memorize these formulas.
depending on what you're working with. But, it's important to note that this is when you see it, you're not lost when you're running your programming, unless you're building your own decision tree code in the back. And we simply have a
log squared of P over P plus N minus N over P plus N times the log squared of N of P plus N. But, let's break that down and see what it actually looks like when we're computing that from the computer script side. Entropy of a target class
of the data set is the whole entropy. So, we have entropy play golf. And when we look at this, if we go back to the data, you can simply count how many yeses and no in our complete data set for playing golf days. In our complete
set, we find we have five days we did play golf and nine days we did not play golf. And so, our I equals, if you add those together, 9 + 5 is 14. And so, our those together, 9 + 5 is 14. And so, our I equals 5 over 14 and 9 over 14. That's
our P and N values that we plug into that formula. And you can go with the 5 over 14 equals .36, 9 over 14 equals .64. And when you do the whole equation, .64. And when you do the whole equation, you get the minus .36 log squared of .36
you get the minus .36 log squared of .36 minus .64 log square root of .64. And we minus .64 log square root of .64. And we get a set value. We get .94. So, we now have a full entropy value for the whole set of data that we're working
with. And we want to make that entropy go down. And just like we calculated the entropy out for the whole set, we can also calculate entropy for playing golf and the outlook. Is it going to be overcast or rainy or sunny? And so, we
look at the entropy we have P of sunny times E of three of two. And that just comes out how many sunny days yes and how many sunny days no over the total, which is five. Don't forget to put the We'll divide that five out later on.
We'll divide that five out later on. Equals P overcast equals 4,0 plus rainy equals 2,3. And then when you do the whole setup, we have 5 over 14. Remember I said there was a total of five. 5 over 14 times the I of three of two plus 4
14 times the I of three of two plus 4 over 14 times the 4,0 and 5 14 over I of 2 3. And so, we can now compute the entropy of just the part that has to do with the forecast. And we get 0.693. Similarly, we can calculate the entropy
of other predictors like temperature, humidity, and wind. And so, we look at the gain outlook. How much are we going to gain from this? Entropy play golf minus entropy play golf outlook. And we can take the original 0.94 for the whole
can take the original 0.94 for the whole set minus the entropy of just the rainy day and temperature, and we end up with a gain of 0.247. So, this is our information gain. Remember we defined entropy and we
defined an information gain. The higher the information gain, the lower the entropy, the better. The information gain other three attributes can be calculated in the same way. So, we have our gain for temperature equals 0.029.
We have our gain for humidity equals 0.152 and our gain for a windy day equals 0.048. And if you do a quick comparison, you'll see the 0.247 is the greatest gain of information, so
that's the split we want. Now, let's build a decision tree. So, we have the outlook, is it going to be sunny, overcast, or rainy? That's our first information gain. And we can continue to go down the tree using the different
information gains with the largest information we can continue down the attribute with the largest information gain as the root node, and then continue to split each sub node with the largest information gain that we can compute.
tongue twister to say all that, you can see that it's a very easy to view visual model. We have our outlook, we split it three different directions. If the play. And then we can split those further down if we want. So, if the
outlook is sunny, but then it's also windy, if it's windy, we're not going to play. If it's not windy, we'll play. So, we can easily build a nice decision tree tomorrow and give us a nice recommendation for the day. So, we want
when it's sunny and windy. Remember the original question that came out? windy. You can see by going down the tree, we go outlook sunny, outlook tomorrow. So, our little smart watch pops up and says, "I'm sorry, tomorrow's
not a good day for golf. It's going to be sunny and windy." And if you're a huge golf fan, you might go, "Uh-oh, it's not a good day to play golf. We can So, we'll sit in front of the TV instead of being out playing golf in the wind.
let's look at the third one of our algorithms we're investigating, support vector machine. Support vector machine is a widely used classification algorithm. The idea of support vector machine is simple. The algorithm creates
a separation line which divides the classes in the best possible manner. For example, dog or cat, disease or no disease. Suppose we have a labeled sample data which tells height and weight of males and females.
know whether it's going to be a male or a female. So, we start by drawing a line. We draw decision lines, but if we consider decision line one, then we will classify the individual as a male. And if we consider decision line two, then
person kind of lies in the middle of the two groups, so it's a little confusing should be under. We need to know which line divides the classes correctly, but how? The goal is to choose a hyperplane and that is one of the key words they
use when we talk about support vector machines. Choose a hyperplane with the greatest possible margin between the decision line and the nearest point within the training set. So, you can see here we have our support vector, we have
the two nearest points to it and we draw a line between those two points. And the distance margin is the distance between the hyperplane and the nearest data point from either set. So, we actually have a value and it should be equally
distant between the two points that we're comparing it to. When we draw the hyperplanes, we observe that line one has a maximum distance. So, we observe that line one has a maximum distance margin, so we'll classify the new data
point correctly. And our result on this one is going to be that the new data point is male. One of the reasons we call it a hyperplane versus a line is that a lot of times we're not looking at just weight and height. We might be
looking at 36 different features or dimensions. And so, when we cut it with a hyperplane, it's more of a three-dimensional cut in the data, multi-dimensional. It cuts the data a certain way and each plane continues to
cut it down until we get the best fit or match. Let's understand this with the You always start with a problem statement when you're going to put some coding now. Classifying muffin and
cupcake recipes using support vector machines. So, the cupcake versus the muffin. Let's have a look at our data set and we have the different recipes here. We have a muffin recipe that has uh so much flour. I'm not sure what
measurement 55 is in, but it has 55. Maybe it's ounces? But uh it has a certain amount of flour, a certain amount of milk, sugar, butter, egg, baking powder, vanilla, and salt. And so, based on these measurements, we
muffin or a cupcake. And you can see in this one, we don't have just two features. We don't just have height and weight as we did before between the male and female. And here, we have a number of features. In fact, in this, we're
looking at eight different features to guess whether it's a muffin or a What's the difference between a muffin and a cupcake? Turns out muffins have more flour, while cupcakes have more butter and sugar. So, basically, the
dessert, where the muffin's a little bit more of a fancy bread. But how do we do that in Python? How do we code that to go through recipes and figure out what the recipe is? And I really just want to say, "Cupcakes versus muffins!" Like
some big professional wrestling thing. Before we start in our cupcakes versus muffins, we are going to be working in Python. There's many versions of Python, many different editors. That is one of the strengths and weaknesses of Python.
Is it just has so much stuff attached to it. It's one of the more popular data science programming packages you can use. In this case, we're going to go ahead and use Anaconda and Jupyter Notebook. The Anaconda Navigator has all
kinds of fun tools. Once you're into the Anaconda Navigator, you can change environments. I actually have a number of environments on here. We'll be using Python 3.6 environment. So, this is in Python version 3.6. Although, it doesn't
matter too much which version you use. I usually try to stay with the 3.x cuz project that's very specifically in version 2.x. 2.7, I think, is usually what most people use in the version two. And then once we're in our Jupiter
Notebook editor, I can go up and create a new file. And we'll just jump in here. In this case we're doing SVM muffin versus cupcake. And then let's start versus cupcake. And then let's start with our packages for data analysis.
We almost always use a couple There's a few very standard packages we use. few very standard packages we use. We use import Oops, import import Numpy That's for number Python. They usually
denote it as NP. That's very common. That's very common. And then we're going to import Pandas as PD. There's a lot of cool things you can do with the Numpy setup as far as
multiplying all the values in an array in a Numpy array data array. Pandas I can't remember if we're using it actually in this data set. I think we do as an import. It makes a nice data frame. And the difference between a data
frame and a Numpy array is that a data frame is more like your Excel spreadsheet. You have columns, you have indexes. So you have different ways of referencing it, easily viewing it. And there's additional features you can run
on a data frame. And Pandas kind of sits on Numpy. So they You need them both in And then finally we're working with the support vector machine. So from SKLearn,
we're going to use the SKLearn model. Import SVM, support vector machine. as a data scientist, you should always try to visualize your data. Some data
obviously is too complicated or doesn't make any sense to the human. But if it's look at it so that you can actually see what you're doing. And for that we're going to use two packages. We're going to import matplotlib.pyplot
as plt. Again, very common. And we're going to import seaborn as sns. And we'll go ahead and set the font scale in the sns right in our import line. That's what this
line. That's what this semicolon followed by a line of data. We're going to set the sns. And these are great cuz the seaborn sits on top of matplotlib library, just like pandas sits on numpy. So, it adds a lot more
obviously not going to get into matplotlib library and seaborn, that'd be its own tutorial. We're really just focusing on the SVM, the support vector machine from scikit-learn. And since we're in Jupyter Notebook,
we have to add a special line in here for our matplotlib library. And that's your percentage sign or ampersand matplotlib library inline. Now, if you're doing this in just a straight code project, a lot of times I
and I'll write it from there. You don't it'll just pop up as its own window on your computer, depending on how your computer set up. Because we're running this in the Jupyter Notebook as a
browser setup, this tells it to display all of our graphics right below on the page. So, that's what that line is for. I remember the first time I ran this, I didn't know that and I had to go look that up years ago. That was quite a
headache. So, matplotlib library inline is just because we're running this on the web setup. And we can go ahead and run this, make sure all our modules are in. They're all in imported, which is great. If you don't have them imported,
pip or however you do it. There's a lot of other install packages out there, have to make sure these are all installed on your Python setup. The next step, of course, is we got to look at the data. Can't run a model for
predicting data if you don't have actual data. So, to do that, let me go ahead and open this up and take a look. And we have our cupcakes versus muffins. And it's a CSV file or .csv meaning that it's comma separated variable.
And it's going to open it up in a nice, uh, spreadsheet for me. And you can see muffin, muffin, muffin, cupcake, cupcake, cupcake. And then it's broken up into flour, milk, sugar, butter, egg, baking powder, vanilla, and salt.
So, what we can do is we can go ahead and look at this data also in our and look at this data also in our Python. We're going to use our pandas module .read
CSV. Remember it's a comma separated variable. And the file name happened to be cupcakes versus muffins. Oops, I got cupcakes versus muffins. Oops, I got double brackets there.
Because the program I loaded or the the place I saved this particular Python program is in the same folder, we can get by with just the file name. But different location, you have to also put down the full path on there.
actually In line you can do this, but let me do the full print. And you can let me do the full print. And you can just type in recipes.head in the Jupyter notebook. But if you're running in code in a different script,
you'd need to go ahead and type out the whole print recipes.head. And pandas knows this is going to do the first five lines of data. And if we flip back on over to the spreadsheet where we opened up our CSV file,
uh, you can see where it starts on line two. This one calls it zero. And then 2 3 4 5 6 is going to match. Go and close that up cuz we don't need that anymore. And it always starts at zero. And these are know, indexes it since we
didn't tell it to use an index in here. So, that's the index number for the left-hand side. And it automatically took the top row as uh labels. So, Pandas using it to read a CSV is just really
slick and fast. One of the reasons we love our Pandas. Not just because love our Pandas. Not just because they're cute and cuddly teddy bears. they're cute and cuddly teddy bears. And let's go ahead and plot our data.
I'm not going to plot all of it. I'm just going to plot the uh sugar and Now, obviously you can see where they get really complicated if we have tons of different features. And so, you'll break them up
and maybe look at just two of them at a time to see how they connect. ahead and use Seaborn. So, that's our SNS. And the command for that is sns.lmplot. And then, the two different variables
And then, the two different variables I'm going to plot is flour and sugar. Data equals recipes. The hue equals type. And this is a lot of fun because it knows that this is Pandas coming in. Um so, this is one of
the powerful things about Pandas mixed with Seaborn and doing And then, we're going to use a palette set one. There's a lot of different sets Seaborn. Or do a regular uh fit regular equals
false. So, we're not really trying to fit anything. And it's a scatter KWS. in Seaborn. Half of these you could probably leave off when you run them. that these were the best settings for doing a Seaborn plot.
And let's go ahead and run that. And because it does it in line, it just puts it right on the page. And you can see right here that just based on sugar and flour alone, there's a definite split.
And we use these models because you can actually look at it and say, "Hey, if I drew a line right between the middle of the blue dots and the red dots, we'd be able to do an SVM and and a hyperplane right there in the middle."
process our data. And we're going to break that up into two parts.
muffin or a cupcake. Well, a computer doesn't know muffin or cupcake. It knows So, what we're going to do is we're going to create a type label. And from this, we'll create a NumPy
np.where And this is where we can do some logic. We take our recipes from our panda and where our type equals muffin, it's going to be zero. And then, if it doesn't equal muffin, which is cupcakes,
it's going to be one. So, we create our type label. our training model, remember we have to have a training data. This is what we're going to train it with is that it's zero or one. It's a muffin or it's not.
recipe features. And if you remember correctly from right up here, the first column is type. So, we really don't need the type column cuz that's our muffin or cupcake. And in
cuz that's our muffin or cupcake. And in pandas, we can easily sort that out. We take our value recipes .columns. That's a pandas function built into pandas.
.values. Converting them to values. So, it's just the column titles going across the top. And we don't want the first one. So, what we do is since it always one. So, what we do is since it always starts at zero, we want one
colon till the end. And then, we want to go ahead and make this a list and this converts it to a list of and this converts it to a list of strings.
look and see what we're looking at for the features. Make sure it looks right. the features. Make sure it looks right. We go ahead and run that. run that. And we can see we have flour, milk, sugar, butter, egg, baking powder,
vanilla, and salt. And that matches what we have up here when we print it out. Everything but the type. So, we have our features and we have our label. Now, the recipe features is just the titles of the columns and we actually
need the ingredients. options. One, we could run it over all the ingredients.
And when you're doing this usually you do, but for our example we want to limit it so you can easily see what's going on. Cuz if we did all the ingredients we have you know, that's what um seven, eight different hyperplanes that
would be built into it. We only want to look at one so you can see what the SVM is doing. And so, we'll take our recipes and we'll do just flour and sugar. Again, you can replace that with your
recipe features and do all of them, but we're going to do just flour and sugar values. We don't need to make a list out of it cuz it's not string values. These are actual values on there. And we can go ahead and
just print ingredients and you can see what that Uh and so, we have just the amount of flour and sugar. Just the two sets of plots.
take this over here and take our recipe features. And so, if we decided to use all the recipe features of different data. So, it just strips out all the labels and everything. We
just have just the values. But, because we want to be able to view this easily in a plot later on, we'll go ahead and take that and just do flour and sugar.
And we'll run that and you'll see it's just the two columns. our model. We'll go ahead and just call it model. And it's a SVM. We're using a package called SVC.
and set the kernel equals linear. So, it's using a specific setup on there. And if we go to the reference on their website for the SVM, you'll see that there's about There's eight of them here. Three of them are
for regression, three are for classification. The SVC, support vector classification, is probably one of the most commonly used. And then there's also one for detecting outliers, and another one that has to do
with something a little more specific on the model. But, SVC and SVR are the two most commonly used, standing for support vector classifier and support vector regression. Remember, regression is an actual value, a float value or whatever
you're trying to work on. And SVC is a classifier, so it's a yes, no, true, false. But, for this, we want to know 0 1, muffin cupcake. We go ahead and create our model. And
once we have our model created, we're going to do model.fit. And this is very their models are followed with the fit command. And what we put into the fit, what we're training with it, is we're putting in
the ingredients, which in this case we limited to just flour and sugar, and the type label, is it a muffin or cupcake? Now, in more complicated data science series, you'd want to split into We
won't get into that today. We split it into a training data and test data. And it into thirds, where a third is used for where you switch between which one's things go into that. It gets very complicated when you get to the higher
end. Not overly complicated, just an extra step, which we're not going to do data. And let's go ahead and run this. And now we have our model fit. And uh I got a error here, so let me fix
that real quick. It's capital SVC, it turns out. I did it lowercase. Support vector classifier. There we go. Let's go ahead and run that.
information that it prints out automatically. These are the defaults of You notice that we changed the kernel to linear. And there's our kernel linear on the printout. And there's other different settings you can mess with.
for right now. For this, we don't really need to mess with any of those. So, next, we're going to dig a little bit into our newly trained model.
show you on a graph. And let's go ahead and get the separating And we're going to say uh we're going to use a W for our variable on here.
And we're going to do model.coefficient_zero. digging into the model. So, we've already got a prediction and a train. This is the math behind it that we're looking at right now.
And so, the W is going to represent two different coefficients. And if you remember, we had Y = MX + C. So, these coefficients are connected to that, but in two-dimensional. It's a
plane. We don't want to spend too much time on this because you can get lost in the confusion of the math. So, if you're a math wiz, this is great. You can go through here and you'll see that we have
through here and you'll see that we have a equals minus w zero over w of one. there. And that's basically the slope that we're generating. And then we're going to do build an xx.
What is xx? We're going to set it up to a numpy array. There's our np.linspace. So, we're creating a line of values between 30 and 60. So, it just
creates a set of numbers for x. And then if you remember correctly, we have our formula y equals the slope times x plus the intercept. Well, to make this work, we can do this as yy
work, we can do this as yy equals the slope times each value in that array. That's the neat thing about numpy. So, when I do a times xx, which is a whole numpy array of values, it multiplies a across all of them.
And then it takes those same values and we subtract the model intercept. That's your uh what we had mx plus c. So, that'd be the what we had mx plus c. So, that'd be the c from the formula y equals mx plus c.
from. A little bit confusing cuz it's digging out of these different arrays. going to take this and we're going to go ahead and plot it. So, plot the parallels to separating hyperplane that pass through the support
vectors. And so, we're going to create b equals a model support vectors. Pull on our support vectors out there. Here's our yy, which we now know is a set of data. And we have we're going to create yy down equals a times xx plus
b1 minus a times b0. And then model support vector b is going to be set that to a new value the minus one set up and YY up equals A times XX
one set up and YY up equals A times XX plus B1 minus A times B0. load these variables up. If you want to know a understand a little bit more of know a understand a little bit more of what's going on, you can see if we print
see it's an array. It's this is a line. It's going to have in this case between 30 and 60, so it's going to be 30 variables in here. variables in here. And the same thing with YY up, YY down.
minute on a graph so you can see what those look like. here and run that so it loads up the variables. Nice clean slate. I'm just going to copy this from before.
Remember this our SNS dot our seaborn plot LM plot flower sugar? what that looks like. It's just a straight graph on there.
And then one of the new things is because seaborn sits on top of pi plot, we can do the pi plot for the line going through. And that is simply plt.plot.
and YY are two corresponding values XY. And then somebody played with this to figure out that the line width equals two and the color black would look nice. thing with the pi plot on there. And you can see when we do this, it's
And you can see when we do this, it's just doing flower and sugar on here. Corresponding line between the sugar and the flower and the muffin versus the flower and the muffin versus cupcake.
Um and then we generated the um support vectors the YY down and YY up. So let's like. So we'll do our PL plot. So we'll do our PL plot. And again this is all against XX the our
And again this is all against XX the our X value. But this time we have YY And let's do something a little fun with this. this. We can put in a K- -
line. And if we're going to do the down one, we also want to do the up one. So here's our YY
up. And when we run that, it adds both sets And so here's our support, and this is what you expect. You expect these two points. So the dash lines go through the nearest muffin and the nearest cupcake
when it's plotting it. And then your SVM goes right down the middle. So it gives it a nice split in our data, and you can see how easy it is to see based just on sugar and flour which one's a muffin or a cupcake.
which one's a muffin or a cupcake. Let's go ahead and create a function to predict muffin or cupcake. I've got my recipes I've pulled off the um
difference between a muffin or a cupcake. And so we need a function to push that through. And uh we'll create a function with def. And let's call it muffin or cupcake. And remember, we're just doing flour and
ingredients. And that actually is a pretty good split. You really don't need and sugar. And let's go ahead and do an if else And let's go ahead and do an if else statement. So if model predict
so we take our model and we do it run a predict. That's very common in SK learn where you have a dot predict. You put the data in, and it's going to return a value. In this case, if it equals zero, then print you're you're at a muffin
recipe. Else, if it's not zero, that means it's recipe. That's pretty straightforward for function or def for definition. d e f is how you do that Python.
function you should run something in it and so let's run a cupcake and we're going to send it values 50 and 20. A muffin or cupcake, I don't know what it gives us. And it says, "Oh, it's a muffin. You're
looking at a muffin recipe." So it very easily predicts whether we're looking at a muffin or a cupcake recipe. Let's plot this. There we go. Plot this actually looks like. And I'm just going to copy and paste it
from below where we're plotting all the points in there. before. If I run it, you'll see it has all the points and the And what we want to do is we want to add another point.
another point. And we'll do plt.plot. for our test we did 50 and 20. And then somebody went in here and decided we'll do a yo for yellow or it's
kind of a orangish yellow color is going to come out. Marker size nine. Those are else played with them to come up with the right setup so it looks good. And you can see there it is graphed. Clearly a muffin.
In this case, in cupcakes versus muffins, the muffin has won. And if you'd like to do your own muffin cupcake contender series, you certainly can send a note down below and the team at Simplilearn will send you over the
cupcake. And that's true of any of the data. We didn't actually run a plot on it earlier. We had men versus women. You can also request that information to run it on your data setup so you can test that out.
So to go back over our setup, we went ahead for our support vector machine code. We did a predict 40 parts flour, 20 parts sugar. I think it was different muffin or a cupcake. Hence, we have built a classifier using SVM, which is
able to classify if a recipe is of a cupcake or a muffin. What's in it for you? We're going to cover clustering. What is clustering? K-means clustering, which is one of the most common used clustering tools out there, including a
flowchart to understand K-means clustering and how it functions. And then we'll do an actual Python live demo on clustering of cars based on brands. Then we're going to cover logistic regression. What is logistic regression?
Logistic regression curve and sigmoid function. And then we'll do another Python code demo to classify a tumor as malignant or benign based on features. And let's start with clustering. Suppose we have a pile of books of different
genres. Now, we divide them into different groups like fiction, horror, education. And as we can see from this young lady, she definitely is into heavy and the maple Canadian leaf on her
education. And we want to go ahead and divide our books up. Well, organizing objects into groups based on similarity is clustering. And in this case, as talking about clustering things with no one categories. But you can also use it
the categories. You just know that you conquer the data and to organize it better. But in this case, we're going to categories. And let's just take a deeper look at that. We're going to use K-means
clustering. K-means clustering is probably the most commonly used clustering tool in the machine learning library. K-means clustering is an example of unsupervised learning. If you remember from our previous thing, it is
used when you have unlabeled data. So, we don't know the answer yet. We have a bunch of data that we want to cluster to different groups. To find clusters in the data based on feature similarity. So, we've introduced a couple terms
here. We've already talked about unsupervised learning and unlabeled data. So, we don't know the answer yet. We're just going to group stuff together and see if we can find an answer of how things connect. We've also introduced
feature similarity. Features being different features of the data. Now, with books, we can easily see fiction and horror and history books, but a lot of times with data, some of that information isn't so easy to see, right,
K-means is one of those tools where we can start finding things that connect, that match with each other. Suppose we have these data points and want to assign them into a cluster. Now, when I look at these data points, I would
just by looking at them. I'd say two of these group of data kind of come together. But in K-means, we pick K clusters and assign random centroids to clusters, where the K clusters represents two different clusters. We
centroids to the clusters, then we compute distance from objects to the centroids. Now, we form new clusters based on minimum distances and calculate the centroids. So, we figure out what the best distance is for the centroid,
then we move the centroid and recalculate those distances. Repeat previous two steps iteratively till the cluster centroids stop changing their positions and become static. Repeat previous two steps iteratively till the
positions become static. Once the clusters become static, then K-means clustering algorithm is said to be converged. And there's another term we see throughout machine learning is converged. That means whatever math
we're using to figure out the answer has come to a solution or it's converged on an answer. Shall we see the flowchart to understand? Make a little bit more sense by putting it into a a nice easy step-by-step. So, we start, we choose K,
we'll look at the elbow method in just a moment. We assign random centroids to clusters, and sometimes you pick the centroids because you might look at the data in a in a graph and say, "Oh, these are probably the central points." Then
we compute the distance from the objects to the centroids. We take that and we form new clusters based on minimum distance and calculate their centroids. Then we compute the distance from objects to the new centroids. And then
we go back and repeat those last two steps. We calculate the distances. So, as we're doing it, it brings into the new centroid, and then we move the centroid around and we figure out what the best which objects are closest to
switch from one centroid to the other as the centroids are moved around. And we continue that until it is converged. Let's see an example of this. Suppose we have this data set of seven individuals and their score on two topics, A and B.
So, here's our subject, in this case referring to the person taking the test. And then we have subject A where we see what they've scored on their first can see what they score on the second subject. Now, let's take two farthest
apart points as initial cluster centroids. Remember, we talked about selecting them randomly or we can also just put them in different points and together. Either one works okay, depending on what kind of data you're
working on and what you know about it. So, we took the two furthest points, one and one and five and seven. And now let's take the two furthest apart points as initial cluster centroids. Each point is then assigned to the closest cluster
with respect to the distance from the centroids. So, we take each one of these distance, and you can see that if we measured each of those distances, and you'd use it the Pythagorean theorem for a triangle in this case because you know
the X and the Y and you can figure out the diagonal line from that or you just That'd be kind of silly, but it would work if you're just eyeballing it. You can see how they naturally come together in certain areas. Now we again calculate
the centroids of each cluster. So cluster one and then cluster two and we look at each individual dot. There's one, two, three were in one cluster. The centroid then moves over. It becomes 1.8 {comma} 2.3. So remember it was at one
and one? Well, the very center of the data we're looking at would put it at the 1. roughly 2 2, but 1.8 and 2.3. And the second one, if we wanted to make the of all the different distances to that centroid, we come up with 4 {comma} 1
and 5 4. So we've now moved the centroids. We compare each individual's distance to its own cluster mean and to that of the opposite cluster and we find and build a nice chart on here that the as we move that centroid around, we now
have a new different kind of clustering of groups and using Euclidean distance between the points and the mean, we get the same formula you see as new formulas dots, distance to the mean centroid of the cluster and distance to the mean
Only individual three is nearer to the mean of the opposite cluster, cluster two, than its own cluster one. And you can see here in the diagram where we've So when we've moved the cluster the centroids of the clusters over, one of
the points shifted to the other cluster because it's closer to that group of individuals. Thus, individual three is relocated to cluster two resulting in a new partition and we regenerate all those numbers of how close they are to
the different clusters. For the new clusters, we will find the actual cluster centroids. So now we move the centroids over and you can see that clusters on here. On comparing the
distance of each individual's distance to its own cluster mean and to that of the opposite cluster, we find that the data points are stable. Hence, we have remember, I brought up a concept earlier, K-means on the K-means
algorithm. Choosing the right value of K will help in less number of iterations. And to find the appropriate number of clusters in a data set, we use the elbow clusters in a data set, we use the elbow method. And within sum of squares, WSS
is defined as the sum of the squared distance between each member of the cluster and its centroid. And so, you see what we've done here is we have the number of clusters. And as you do the same K-means algorithm over the
what that centroid looks like, and you find the optimal, you can actually find the optimal number of clusters using the elbow. The graph is called as the elbow method. And on this, we guessed at two just by looking at the data. But as you
can see, the slope You actually just look for right there where the elbow is answer that we want two different to start with K-means equals two. A lot of times, people end up computing K-means equals two, three, four, five until they
find the value which fits on the elbow joint. Sometimes, you can just look at the data, and if you're really good with that specific domain Number domain, I that that where to pick those numbers or where to start guessing at what that K
value is. So, let's take this. And we're going to use a use case using K-means clustering to cluster cars into brands using parameters such as horsepower, cubic inches, make, year, etc. So, we're going to use the data set cars data
having information about three brands of cars, Toyota, Honda, and Nissan. And we'll go back to my favorite tool, the Anaconda Navigator with the Jupyter Notebook. And let's go ahead and flip over to our Jupyter Notebook. And in our
Jupyter Notebook, I'm going to go ahead and just paste the basic code that we We're not going to go too much into this code because we've already discussed NumPy, we've already discussed Matplotlib Library and Pandas. NumPy
being the number array and Pandas being the Pandas data frame and Matplotlib for the graphing. And don't forget since if you're using the Jupyter Notebook, you do need the Matplotlib Library inline so that it plots everything on the screen.
If you're using a different Python editor, then you probably don't need that cuz they'll have a pop-up window on your computer. And we'll go ahead and run this just to load our libraries and our setup into here. The next step is of
course to look at our data, which I've already opened up in a spreadsheet. And you can see here we have the miles per gallon, cylinders, cubic inches, horsepower, weight pounds, how you know how heavy it is, time it takes to get to
60. My car is probably on this one at about 80 or 90. What year it is. So this is you can actually see this is kind of older cars and then the brand, Toyota, Honda, Nissan. So the different cars are coming from all the way from 1971 if we
scroll down to the '80s. So we have between the '70s and '80s the number of cars that they've put out. And let's we come back here, we're going to do importing the data. So we'll go ahead and do data set equals and we'll use
Pandas to read this in and it's from a CSV file. Remember you can always post this in the comments and request the data files for these, either in the comments here on the YouTube video or go to simplylearn.com and request that. The
cars.csv I put it in the same folder as the code that I've stored. So my Python code is stored in the same folder so I don't have to put the full path. If you have to change this and double-check your name variables. And we'll go ahead
and run this and we've chosen data set arbitrarily cuz you know it's a data set we're importing. And we've now imported our cars.csv into the data set. So we're going to create the X data. This is the one that we're going to try
to figure out what's going on with. And then there is a number of ways to do this, but we'll do it in a simple loop so you can actually see what's going on. so you can actually see what's going on. So we'll do for I in x.columns.
columns and a lot of times it's important I'll I'll make lists of the columns and do this because I might remove certain columns or there might be columns that I want to be processed differently. But
for this, we can go ahead and take x of I and we want to go fill in a and that's a pandas command. But the question is what are we going to fill the missing data with? We definitely don't want to just put in a number that doesn't
actually mean something. And so one of the tricks you can do with this is we can take x of I and then in addition to that we want to go ahead and turn this into an integer cuz a lot of these are integers. So we'll go ahead and keep it
integers and let me add the bracket here and a lot of editors will do this. bracket. Make sure you get that second bracket in there if it's a double bracket. That's always something that happens regularly. So once we have our
integer of x of I, this is going to fill in any missing data with the average. And I was so busy closing one set of brackets, I forgot that the mean is also we can see here we're going to fill in all the data with the average value for
that column. So if there's missing data it's in the average of the data it does have. And then once we've done that, we'll go ahead and loop through it again and just check and see to make sure everything is filled in correctly. And
everything is filled in correctly. And we'll print and then we take x is null and this returns a set of the null value or the how many lines are null and we'll just sum that up to see what that looks like. And so when I run this, and so
with the x, what we want to do is we want to remove the last column because trying to see if we can cluster these things and figure out the models. There is so many different ways to sort the x out. For one, we could take the X
and we could go data set, our variable we're using, and use the I location, one of the features that's in Pandas. And we could take that and then take all the rows and all but the last column of the data set.
And at this time we could do values. We just convert it to values. So, that's one way to do this. And if I let me just put this down here and print X. It's a capital X we chose. And I run this, you can see it's just the values. We could
also take out the values and it's not going to return anything cuz there's no values connected to it. What I like to do with this is instead of doing the I location, which does integers, more common is to come in here and we have
our data set and we're going to do data set dot or data set dot columns. And remember that list all the columns. So, if I come in here
if I come in here let me just mark that as red and I print let me just mark that as red and I print data set dot columns. have my MPG, cylinders, everything, including the brand, which we don't
want. So, the way to get rid of the brand would be to do data columns of everything but the last one, minus one. So, now if I print this, you'll see the And so, I can actually just take data
set columns minus one and I'll put it right in here for the columns we're going to look at. And uh let's unmark this and unmark this. And now if I do an X dot head
I now have a new data frame and you can see right here we have all the different of the year. And it turns out when you start playing with the data set, you're going to get an error later on and it'll say, "Cannot
an error later on and it'll say, "Cannot convert string to float value." And things, the way they recorded them, must have been recorded as strings. So, we have a neat feature in here on Pandas to convert. And it is simply convert
convert. And it is simply convert objects. Oops, convert {underscore} numeric numeric equals true. And yes, I
have it memorized the convert numeric in there. If I'm working with a lot of these things, I remember them, but um depending on where I'm at, what I'm doing, I usually have to look it up. And we run that. Oops, I must have missed
something in here. Let me double-check my spelling. And when I double-check my underscore in the convert objects. And when I run this, it now has everything when I run this, it now has everything converted into a numeric value, cuz
that's what we're going to be working with is numeric values down here. And the next part is that we need to go through the data and eliminate null values. Most people when they're doing small amounts working with small data
a null value, and they have to go back and do this. So, you know, be aware, things are going to pop up, and sometimes you go backwards to fix it.
exploring the data and understanding what you have. And I should have done this earlier, but let me go ahead and increase the size of let me go ahead and increase the size of my window one notch.
So, we'll do for I in working with X {dot} columns, we'll page through all {dot} columns, we'll page through all the columns, and we want to take X of I, and we're going to change that, and we're going to alter it. And so, with
this, we want to go ahead and fill in X of I. Pandas has the fill NA. And that just fills in any non-existent missing data. And we'll put my brackets up, and there's a lot of different ways to fill
If you have a really large data set, some people just void out that data cuz if and then look at it later in a separate exploration of data. One of the tricks we can do is we can take our column
and we can find the means. And the means is in other uh quotation So, when we take the columns, we're going to fill in the the non-existing one with the means. The problem is that returns a decimal float. So, some of
these aren't decimals. Certainly, maybe a little careful of doing this, but for this example, we're just going to fill it in with the integer version of this. It keeps it on par with the other data that isn't a decimal point.
A lot of times you do this first part first to double-check, then you do the make sure you did it right. So, we're going to go through and test for missing data. And one of the ways you can do that is
simply go in here and take our X of I column. So, it's going to go through the X of I column and says is null. So, it's going to value, it actually goes through the all the rows of each column is null, and
then we want to go ahead and sum that. So, when we take that, we add the sum value. These are all pandas, so is null is a panda command, and so is sum. And if we go through that, we go ahead and run it.
you'll see that all the columns have zero null values. So, we've now tested and clean. We have no null values. Everything is now a number value. We turned it into numeric. And we've removed the last column in our data.
And at this point, we're actually going to start using the elbow method to find the optimal number of clusters. So, we're now actually getting into the SK learn part. Uh the K means clustering on here.
one more notch so you can see what I'm typing in here. cluster I'm going to import K
means. I always forget to capitalize the K and the M when I do this. So, it's capital K, capital M, K means. And we'll go ahead and create a um array WCSS
equals Let me get an empty array. If you remember from the elbow method from our slide within the sums of squares, WSS is defined as the sum of squared distance between each member of the cluster and
its centroid. So, we're looking at that change in differences as far as a squared distance. And we're going to run this over a number of K mean values. In fact, let's go for I in range. We'll do 11 of them.
Range 0 of 11. And the first thing we're going to do is we're going to create the actual We'll do it all lower case.
object from the K means we just imported. And the variable that we want to put into this is in clusters. And we're going to set that equals to I. That's the most important
one cuz we're looking at how increasing the number of clusters changes our answer. There are a lot of settings to the K means. Our guys in the back did a great job just kind of playing with some of them.
The most common ones that you see in a lot of stuff is how you would knit your K-means. So, we have K-means plus plus plus. This is just a tool to let the model itself be smart how it picks the
centroids to start with its initial centroids. We only want to iterate no more than 300 times. We have a max iteration we put in there. We have an the n_init, the random_state equals zero. You really don't need to
first learning this. As you start digging in deeper, you start finding that these are shortcuts that will speed up the process that we're working with is the n_clusters equals i. So, we're going to
literally train our K-means 11 times. We're going to do this process 11 times. And if you're working with a big data, you know, the first thing you do is you run a small sample of the data so you can test all your stuff on it. And you
can already see the problem that if I'm going to iterate through a terabyte of data 11 times, and then the K-means itself is iterating through the data multiple times, that's a heck of a process. So, you got to be a little
careful with this. A lot of times though, you can find your elbow, using number on a sample of data, especially if you're working with larger data So, we want to go ahead and take our K-means and we're just going to fit it.
learn, very common, that you fit your model. And if you remember correctly, X. And once we fit this value, we go back to the array we made, and we want to go ahead and just append that value on the
And it's not the actual fit we're putting in there. It's when it generates it, it generates the value you're looking for is inertia. So, K-means.inertia will pull that specific value out that we need.
And let's get a visual on this. We'll do our plt.plot, and what we're plotting here is first the X axis, which is range 0 to 11. So, that will generate a nice little plot there. And the WCSS for our Y axis.
title. And let's see, we'll just give it the elbow method for the title. And let's get some labels. So, let's go ahead and x label. And we'll we'll do number of clusters
And we'll we'll do number of clusters for that. And plt y label. And for that we can do oops, there we go. WCSS since that's what we're doing on the plot on there. And finally, we want to go ahead and display our graph,
which is simply plt. oops. .show. There we go. And because we have it set to inline, it'll appear inline. Hopefully, I didn't make a type error on there.
graph. You can see a very nice elbow joint there at two. And again, right around three and four. And then after that, there's not very much. Now, as a data scientist, if I was looking at this,
actually try both of them to see what the output looked like. And they've we're just going to use three as a setup that looks like when we actually use this to show the different kinds of
this to show the different kinds of cars. means to the cars data set. And basically, we're going to copy the code that we looked through up above, where K means equals K means number of clusters.
And we're just going to set that number of clusters to three. for. And you could do three and four on this and graph them just to see how they curious to look at that. But for this, we're just going to set it to three. Go
go and create our own variable, Y K means for our answers. And we're going to set that equal to whoops, I did a double equal there, to K But we're not going to do a fit. We're going to do a fit predict is the set up
you want to use. And when you're using untrained models, you'll see a slightly then you see just the predict, but we want to both fit and predict the K means on this. And that's fit underscore predict and then our capital X is the
data we're working with. And before we plot this data, we're going to do a little Pandas trick. We're going to take our X value and we're going to set X as matrix. So we're converting this into a nice
rows and columns kind of set up, but we want the we're going to have columns equals none, so it's just going to be a matrix of data in here. And let's go ahead and run that. It's a little warning. You'll see these
warnings pop up cuz things are always being updated. So there's like minor changes in the versions in future versions. Instead of matrix, now that it's more common to set it dot values instead of doing as matrix. But as
matrix works just fine for right now and you'll want to update that later on. But let's go ahead and dive in and plot this and see what that looks like. And before we dive into plotting this data, I always like to take a look and
see what I am plotting. So let's take a look at Y out down here. And we see we have an array of answers. And we see we have an array of answers. We have 2 1 0 2 1 2. So it's clustering
rows of data based on the three different spaces it thinks it's going to be. And then let's go ahead and print X and see what we have for X. And we'll see that X is an array, it's a matrix. So we
have our different values in the array. And what we're going to do, it's very hard to plot all the different values in the array. So we're only going to be looking at the first two or at position zero and one.
And if you were doing a full presentation in front of the board you might actually do a little different than and dig a little deeper into the different columns we looked at. But, we'll only look at columns one and two
So, let's go ahead and clear this data out of here and let's bring up our plot. We're going to do a scatter plot here. So, PLT scatter This looks a little complicated. So,
let's explain what's going on with this. We're going to take the X values and we're only interested in Y of K means equals zero, the first cluster. And then we're going to take value zero for the X axis and then we're going to
interested in K means equals zero, but we're going to take the second column. So, we're only looking at the first two columns in our answer or in the data. And then the guys in the back played
pretty. And they discovered that it looks good with as a size equals 100, that's the size of the dots. We're going to use red for this one and when they were looking at the data and
Toyota on this. We're just going to go ahead and label it Toyota. Again, that's something you really have to explore in here as far as playing with those go ahead and hit enter in there and I'm
just going to paste in the next two lines, which is the next two cars. And this is our Nissan and Honda. And you'll see with our scatter plot we're now looking at Y underscore K means equals one.
And we want the zero column and Y K means equals two. Again, we're looking at just the first two columns, zero and one. And each of these rows then one. And each of these rows then corresponds to Nissan and Honda.
And I'll go ahead and hit enter on there. And uh finally, let's take a look and put the centroids on there. Again, we're going to do a scatter plot. And on the centroids, you can just pull that from our K-means, the uh model we
created, dot cluster centers, and we're going to just do um all of them in the first number and all of them in the second number, which is 0 of them in the second number, which is 0 1, cuz you always start with 0 and 1.
and everything to make it look good. We'll do a size of 300. We're going to them uh so it's good to have some good labels, centroids. And then we do want to do a title, PLT title.
And pop up there. PLT title, cuz you always make want to make your graphs look pretty, and we'll call it clusters of car make. And one of the features of the plot library is you can add a
legend. It'll automatically bring in it since we've already labeled the different aspects of the legend with Toyota, Nissan, and Honda. show. So we can actually see it. And remember,
it's in line. Uh so if you're using a different editor that's not the Jupiter notebook, you'll get a pop-up of this. And you should have a nice set of clusters here. So we can look at this and we have uh clusters of Honda in
and we have uh clusters of Honda in green, Toyota in red, Nissan in purple. And you can see where they put the centroids to separate them. Now, when we're looking at this, we can also plot a lot of other different data
on here as far cuz we only looked at the first two columns. This is just column one and two, or 0 1 as as you label them in computer scripting. But you can see make, and we've been able to pull out the data, and you can see how just these
two columns form very distinct clusters of data. might take a look and say, "Well, what makes these different?" Almost going in reverse, you start looking at the data and pulling apart the columns to find
out why is the first group set up the way it is. Maybe you're doing loans and not defaulting on their loans?" And why is the last group defaulting on their loans? And why is the middle group 50% defaulting on their bank loans? And you
start finding ways to manipulate the data and pull out the answers you want. K-means for clustering, let's move on to the next topic.
Now, let's look into logistic regression. The logistic regression algorithm is the simplest classification algorithm used for binary or multi-classification problems. And we can see we have our little girl from
Canada, who's into horror books, is back. That's actually really scary when eyes. In the previous tutorial, we learned about linear regression, dependent and independent variables. So, to brush up, y = mx + c. Very basic
to brush up, y = mx + c. Very basic algebraic function of y and x. The dependent variable is the target class variable we are going to predict. variable we are going to predict. The independent variables, x1 to all the
way up to xn, are the features or attributes we're going to use to predict the target class. We know what a linear regression looks like, but using the graph, we cannot divide the outcome into categories. It is really hard to
categories. It is really hard to categorize 1.5, 3.6, 9.8. Uh for example, a linear regression graph can tell us that with increase in number of hours studied, the marks of a student will increase. But, it will not tell us
whether the student will pass or not. In such cases, where we need the output as categorical value, we will use logistic regression. And for that, we're going to use the sigmoid function. So, you can see here we have our marks, 0 to 100,
number of hours studied. That's going to be what they're comparing it to in this example. And we use the formula line that says y = mx + c. And when we use that says y = mx + c. And when we use the sigmoid function, we have p = 1 / 1
the sigmoid function, we have p = 1 / 1 + e to the - y. It generates a sigmoid curve. And so, you can see right here when you take the ln, which is the natural logarithm. I always thought it should be nl, not ln. That's just the
inverse of uh e, your e to the - y. And so, when we do this, we get ln of p over so, when we do this, we get ln of p over 1 - p = m * x + c. That's the sigmoid curve function we're looking for. And we can zoom in on the function, and you'll
see that the function as it derives goes to 1 or to 0, depending on what your x value is. And the probability, if it's greater than 0.5, the value is automatically rounded off to 1, indicating that the student will pass.
studying, they will probably pass. Then you have a threshold value at the 0.5, the middle usually. And your probability, if it's less than 0.5, the value is rounded off to 0, indicating the student will fail. So, if they're
going to fail. This, of course, is ignoring the outliers of that one doesn't need any studying to memorize everything. That's not me, unfortunately. I have to study hard to learn new stuff.
Problem statement: to classify whether a tumor is malignant or benign. And this is actually one of my favorite data sets to play with because it has so many features. And when you look at them, you really are hard to understand. You can't
So, it gives you a chance to kind of dive into what data looks like when you domain of the data. But I also want you to remind you that in the domain of medicine, if I told you that my probability was really good at
probability was really good at classifying things that say 90% or 95% and I'm classifying whether you're going to have a malignant or a benign tumor. it tested anyways. So, you got to remember the domain we're working with.
know you're just going to go get a biopsy because, you know, it's that Just referencing the domain, it's important. It might help the doctor know where to look just by understanding what kind of tumor it is. So, it might help
them or aid them in something they missed from before. So, let's go ahead back to the domain part of it in just a minute. So, use case and we're going to do our normal imports here. We're importing NumPy, Pandas, Seaborn, the
Matplotlib library and we're going to do Matplotlib library in line since I'm let's go ahead and flip over there and get this started. So, I've opened up a new window in my Anaconda Jupyter notebook.
don't have to use Anaconda for the Jupyter notebook. I just love the interface and all the tools in Anaconda brings. So, we got our import NumPy as NP for our NumPy number array. We have
our Pandas PD. We're going to bring in Seaborn to help us with our graphs as SNS. So many really nice tools in both Seaborn and Matplotlib library. Then we'll do our Matplotlib library.pyplot as PLT and then of course we want to let
ahead and just run that so it's all set up. data. I'm not creative today. Uh equals PD and not creative today. Uh equals PD and this happens to be in a CSV file.
So, we'll use a pd.read_csv and I happen to name the file or renamed it data for p2.csv. You can of course write in the comments data set itself or go to the Simply Learn website and we'll be happy to
supply that for you. And let's just open up the data before we go any further and let's just see what it looks like in a spreadsheet. So, when I pop it open in a local spreadsheet, this is just a CSV file,
comma separated variables, we have an ID. So, I guess the categorizes for reference for what ID which test was done. The diagnosis, M for malignant, B options on there. And as we were going to try to predict is the M and B and
tested. And then we have like the radius mean or average, the texture average, perimeter mean, area mean, smoothness. I don't know about you, but unless you're stuff, I mean, you can guess what concave means just by the term concave,
means in the measurements they're taking. So, they have all kinds of stuff like how smooth it is, the symmetry. And these are all float quick and you'll see there's, I believe, 36 if I remember correctly in this one.
take when they go in there and they take a look at the different growth, the tumorous growth. So, back in our data, and I put this in the same folder as the code. So, I saved this code in that folder. Obviously, if
want to put the full path in there. And we'll just do um Pandas first five lines of data with the data.head. We run that, we can see that we have
pretty much what we just looked at. We have an ID, we have a diagnosis. If we go all the way across, you'll see all the different columns coming across all the different columns coming across displayed nicely for our data.
seaborn, which we referenced as SNS, makes it very easy to go in here and do a joint plot. You'll notice the very similar to cuz it is sitting on top of the plot library. So, the joint plot does a lot of work for us. And we're
just going to look at the first two columns that we're interested in, the radius mean and the texture mean. We'll just look at those two columns. And data equals data. So, that tells it which two columns we're plotting and
then we're going to use the data that we pulled in. Let's just run that. here. And there's all kinds of cool things on the texture mean and the radius mean. Obviously, the axes. You can also see
show that for the radius mean. Where is the most common is radius mean come up and where are the most common texture is. That's what we're looking at the the on each growth it's average texture and on each radius it's average radius on
there. Gets a little confusing cuz we're talking about the individual objects' average and then we can also look over here and see the the histogram showing us the median or how common each measurement is. And that's only two
columns. So, let's dig a little deeper into seaborn. They also have a heat map. And if you're not familiar with heat maps, a heat map just means it's in color. That's all that means. Heat map I guess the original ones were plotting
heat density on something and so ever since then it's just called a heat map. our corresponding numbers to put that into the heat map. And that's simply data.corr for that. That's a pandas expression.
data frame. So, that's one of the cool tools in pandas for our data. And let's just pull that information into a heat map and see what that looks like. And you'll see that we're now looking at all the different features. We have our
ID, we have our texture, we have our area, our compactness, concave points. And if you look down the middle of this chart diagonal going from the upper left to bottom right, it's all white. That's because when you
compare texture to texture, they're identical. So, they're 100% or in this case, perfect one in their correspondence. And you'll see that when you look at, say, area or right below it, it has
almost a black on there when you compare it to texture. So, these have almost no form a linear graph or something that you can look at and say that they're all scattered data. This is really just a really nice graph
to get a quick look at your data. Doesn't so much change what you do, but an answer or something like that or you start looking at some of these individual pieces, you might go, "Hey, that doesn't match according to showing
our heat map, this should not correlate with each other. And if it is, you're why? What's going on? What else is coming in there?" information on here. I mean, we can see from the ID,
there's no real one feature that just says, if you go across the top line, that lights up. There's no one feature that says, "Hey, if the area's a certain malignant." It says there's some that sort of add up. And that's a big hint in
the data that we're trying to ID this, whether it's malignant or benign, that's go, "Okay, we can't solve this with any one feature. It's going to be something that includes all the features or many of the
solution for it. And while we're exploring the data, let's explore one exploring the data, let's explore one more area and let's look at data.isnull. We want to check for null values in our data. If you remember from earlier in
differently where we added stuff up and summed them up. You can actually with pandas do it really quickly, data.isnull and sum it, and it's going to go across all the columns. So, when I run this,
you're going to see all the columns come up with no null data. few steps, we've done a lot of exploration. We have
looked at the first two columns and seen how they plot with the seaborn with a joint plot, which shows both the histogram and the data plotted on the XY coordinates. And obviously, you can do that more in
with different columns and see how they plot together. And then we took and did the seaborn heat map, the sns.heatmap of the data. And you can see right here where it did a nice job showing us some
bright spots where stuff correlates with each other and forms a very nice combination or points of scattering points. And you can also see areas that don't. And then finally, we went ahead and
checked the data. Is the data null value? Do we have any missing data in there? Very important step cuz it'll crash later on if you forget to do this step, it will remind you when you get that nice error
code that says null values, okay? So, not a big deal if you miss it, but it it's no fun having to go back when you're when you're in a huge process and 10 steps later and you got to go remember where you were pulling the data
in. So, we need to go ahead and pull out our X and our Y. So, let me just put that down here. And we'll set the X equal to And there's a lot of different options here. Certainly, we could do X equals
all the columns except for the first two cuz if you remember, the first two is the ID and the diagnosis. So, that certainly would be an option. But what we're going to do is we're actually going to focus on the worst the
worst radius, the worst texture, parameter, area, smoothness, compactness, and so on. One of the reasons to start dividing your data up when you're looking at this information is sometimes the data will be the same
data coming in. So, if I have two measurements coming in to my model, it might overweigh them. It might overpower the other measurements because it's information in twice. That's a little bit past the scope of
this tutorial. What I want you to take away from this though is that we are our team in the back went ahead and said, "Hey, let's just look at the worst." So, I'm going to create a an array,
and you'll see this array radius worst, texture worst, perimeter worst. We've I'm just going to put that in my X. So, this X is still a pandas data frame, but it's just those columns. And our Y, if you remember correctly, is going to be
Oops, hold on 1 second. It's not X, it's data. There we go. So, X equals data, and then it's a list of the different columns, the worst of the worst. And if to have our answer for our Y, for the stuff we know.
And if you remember correctly, we're just going to be looking at the diagnosis. That's all we care about is what is it diagnosed? Is it benign or malignant? And since it's a single column, we can
just do diagnosis. Oh, I forgot to put the brackets over the There we go. Okay. So, it's just diagnosis on there, and we can also real quickly do like X.head if you want to see what that looks like, and Y.head,
and run this. And you'll see um it only does the last one. I forgot about that if you don't do print. You can see that the the Y.head is just MMM cuz the first ones are all malignant. And if I run this, the X.head
is just the first five values of radius worst, texture worst, parameter worst, area worst, and so on. I'll go ahead and take that out. I'll go ahead and take that out. So, moving down to the next step, we've
built our two data sets, our answer and then the features we want to look at. In data science, it's very important to test your model.
So, we do that by splitting the data. And from SK-Learn model selection, we're going to import train test split. So, we're going to split it into two groups. There are so many ways to do this. I noticed in one of the more modern ways
actually split it into three groups and then you model each group and test it all kinds of and there's reasons for that, which is past the scope of this and for this particular example isn't necessary. For this, we're just going to
split it into two groups, one to train our data and one to test our data and our data and one to test our data and the SK-Learn uh {dot} model selection, we have train test split. You could write your own quick code to do this
up into two groups, but they do it for us nicely. actually do it in one statement with this where we're going to generate four this where we're going to generate four variables, capital X train, capital X
test. So, we have our training data we're going to use to fit the model and then we have our Y train, so we're going to train the answer and then we have our test. So, this is the stuff we want to see how good it did on our model.
And we'll go ahead and take our train test split that we just imported. And we're going to do X and our Y, our two different data that's going in for our split. And then the guys in the back came up
and wanted us to go ahead and use a test size equals {point}3. That's test {underscore} size. Random state, it's always nice to kind of switch your random state around, but not that important. What this means is that the
test size is we're going to take 30% of the data and we're going to put that into our test variables, our Y test and our X test. And we're going to do 70% into the X train and the Y train. So, we're going to use 70% of the data to
train our model and 30% to test it. Let's go ahead and run that and load those up. So, now we have all our stuff split up and all our data ready to go. part. We're going to actually going to do our create our model.
So, let's go ahead and bring that in from SKlearn. We're going to bring in our linear model and we're going to import logistic regression. That's the actual model we're using. And this is going to call that log model.
Oops, there we go. Model and let's just set this equal to our logistic regression that we just imported. So, now we have a variable log model set to that class for us to use. And with most the models in the SKlearn,
we just need to go ahead and fix it fit do a fit on there and we use our X train that we separated out with our Y train. And let's go ahead and run this. So, once we've run this, we'll have a model that fits this data, that's 70% of our
that fits this data, that's 70% of our training data. that you can set on there. There's a lot of different choices you can make, but to let all the default set. We don't really need to mess with those on this
particular example. And there's nothing in here that really stands out as super important until you start fine-tuning it. But for what we're doing, the basics will work just fine. And then let's We need to go ahead and test out our model.
Is it working? So, let's create a variable Y predict. And this is going to variable Y predict. And this is going to be equal to our log model. And we want to do a predict. Again, very standard format for the SKlearn library
is taking your model and doing a predict on it. And we're going to test Y predict against the Y test. So, we want to know what the model thinks it's going to be. That's what our Y predict is. And with that, we want the capital X X test. So,
we have our train set and our test set and now we're going to do our Y predict. And let's go ahead and run that. And if we print Y predict Let me go ahead and run that.
You'll see it comes up and it predicts an I prints a nice array of B and M for benign and malignant. For all the different test data we put So, that's pretty good. We're not sure exactly how good it does, but we can see
that it actually works and is functional. It was very easy to create. You'll always discover with our data science that as you explore this, you prepping your data and making sure your data coming in is
good. There's a saying, "Good data in, good answers out. Bad data in, bad answers out." That's only half the thing. That's only half of it. Selecting your models becomes the next part as far as how good
fine tuning it depending on what model you're using. good this came out. So, we have our Y predict here log model dot predict X predict here log model dot predict X test.
we're going to go from the SK learn dot metrics. We're going to import classification report. And that just reports how good our model is doing and then we're going to feed it the model data and let's just print this out.
data and let's just print this out. And we'll take our classification report And we'll take our classification report and we're going to put into there
what we actually know is true and our prediction, what our model side. And let's run that and see what that And let's run that and see what that does.
have a precision for benign and malignant B and M. And we have a precision of 93 and 91, a total of 92. So, it's kind of the average between these two of 92. There's
all kinds of different information on here. Your F1 score, your recall, your support coming through on this. flip back to our slides that they put together for describing it. And so, here
we're going to look at the precision using the classification report. And you above. Some of them my numbers might be different cuz it does randomly pick out which data we're using. So, this model is able to predict the type of tumor
with 91% accuracy. So, we look back here, that's you will see where we have benign and malignant, it actually is 92 coming up here. But, we're looking about a 92, 91% precision. And remember, I reminded you
about domain. So, we're talking about the domain of a medical domain with very catastrophic outcome, you know, at 91 or 92% precision, you're still going to go in there and have somebody do a biopsy on it. Very different than if you're
investing money and there's a 92% chance you're going to earn 10% and 8% chance you're going to lose 8%, you're probably going to bet the money cuz at that odds, money and in the long run, if you do that enough, you definitely will make
money. And also with this domain, I've actually seen them use this to identify different forms of cancer. That's one of the things that they're starting to use these models for cuz then it helps the doctor know what to investigate. So,
that wraps up this section. We're finally we're going to go in there and let's discuss the answer to the quiz asked in machine learning tutorial part Can you tell what's happening in the following cases? Grouping documents into
different categories based on the topic and content of each document. This is an clustering can be used to group the documents by topics using bag of words approach. So, if you got in there that
hopefully you had at least one or two examples like K means that are used for clustering different things, then give yourself a two thumbs up. B, identifying handwritten digits in images correctly. This is an example of classification.
would be to extract digit dependent features like curvature of different digits, etc. And then use a classifier like SVM to distinguish between images. Again, if you got the fact that it's a classification example, give yourself a
thumb up. And if you're able to go, "Hey, let's use SVM or another model for on it. C, behavior of a website indicating that the site is not working as designed.
This is an example of anomaly detection. In this case, the algorithm learns what is normal and what is not normal, usually by observing the logs of the website. Give yourself a thumbs up if you got that one. And just for a bonus,
anomaly detection? One of the ones I use it for in my own business is detecting anomalies in stock markets. Stock markets are very fickle, and they behave very erratical. So, finding those erratic areas and then finding ways to
track down why they're erratic. Was something released in social media? Was something released You can see where knowing where that anomaly is can help you to figure out what the answer is to it in another area. D, predicting salary
of an individual based on his or her years of experience. This is an example of regression. This problem can be mathematically defined as a function between independent years of experience and dependent variables salary of an
individual. And if you guessed that this was a regression model, give yourself a thumbs up. And if you're able to remember that it used between independent and dependent variables and that terms, give yourself two thumbs up.
Summary. So, to wrap it up, we went over what is K-means, and we went through also the chart of choosing your elbow method and assigning a random centroid to the clusters, computing the distance, and then going in there and figuring out
through that loop until it gets the perfect centroid. And we looked into the elbow method to choose K based on running our clusters across a number of variables and finding the best location
for that. We did a nice example of clustering cars with K-means, even columns to make it simple and easy to graph. You can easily extrapolate that and look at all the different columns and see how they all fit together. And
we looked at what is logistic regression. We discussed the sigmoid function, what is logistic regression, and then we went into an example of classifying tumors with logistics. I hope you enjoyed part two of machine
&gt;&gt; In this video, we are going to learn how to automate Google Sheets using n8n. It is a very powerful workflow automation tool. We will start by setting up a webhook trigger to receive data from external sources, then connect it to
Google Sheets to automatically store and update data in real time. By the end of this tutorial, guys, you will be able to capture form submissions, automate data entry without manual work, or set up live workflow that runs seamlessly. So,
whether you are a beginner or an automation pro, this tutorial is a must-see for you. First, understanding what exactly is n8n. So, I'm at the what exactly is n8n. So, I'm at the official website of n8n, and n8n is an
open-source workflow automation tool that allows you to connect various applications like APIs and databases without writing extensive code. It provides a visual editor where you can create workflows by linking different
nodes. Think of it an alternative to Zapier and Apache Airflow. But this tool has more control and flexibility and also it has self-hosting capabilities. Now if you see the key features of n8n are as follows. It has this amazing
visual editor where you can drag and drop nodes to create workflows. No complex coding is required. It has over 350 plus integrations and it supports APIs, databases, messaging apps, AI tools and cloud services. For example,
you can think of like Google Sheets, Slack, Telegram, OpenAI, AWS, GitHub, MySQL. So it has integration to these tools. And apart from that, it has self-hosting and cloud options. You can run it on your own server for data
privacy, also available as cloud service. It has also conditional logic and data transformation. It supports if condition, loops, branching. You could also modify API responses, filter data
and perform calculations. And guys, this feature is very, very amazing. For real-time automation, it has webhooks. So n8n can receive data from webhooks to trigger instant actions. For example, when a form is submitted in Typeform,
I'll send an email via Gmail. So it can automate in real time. Now you will be wondering, how does n8n works? Now, let us try to understand how n8n works exactly. So you can think of n8n as a digital
assistant that automates your task without requiring you to write complex codes. It basically works by connecting different application, services and database into sequence of steps called workflow. So if you go and login to the
register for this n8n, you you're going to get a workflow template. Okay, you can go all over here. And you can
and give a description about it and this could be your you know template code whether it is free or paid you can just click on this and after that I have reviewed my submission guidelines of N8N terms and services and then you can
click this workflow. Now let us try to understand what exactly is a workflow. But before that let us try to understand nodes, okay? So nodes are tiny workers that do specific task. You can consider
this something like that. And workflows are a sequence of nodes working together to complete a job. Like for example let us say we are sending an email when you get a new form submission. Imagine you have a Google form where people can
contact you. So every time someone builds the form you want to automatically send them an email. So in N8N you can create a workflow like this. So a Google Forms node collects the new response and Gmail node sends an
automatic email to the person who submits the form. So this is something like this. We'll be doing a one hands-on tutorial also to elaborate more about it. So this is one of the way how it works. Next there's a concept of
triggers. So there are trigger-based execution mode also. How to start your workflow. So you have to also understand guys that N8N workflows don't run all the time. They only start when something happens. So there are three ways to
trigger a workflow. So the first one is trigger-based execution, okay? Which is basically called as an event-driven. The workflow automatically starts when a specific event occurs. For example let's
say a new row is added to the Google Sheets. Now the workflow starts to run. Or you can think about it something like that when a customer fills out a Typeform survey the workflow runs. Think of it like a doorbell. When someone
rings a bell like for example someone submits a form or updates a database, the workflow wakes up and does its jobs. Next one, we have the schedule execution, or we call it as cron jobs.
Now, here the workflow run on schedule, like daily, weekly, monthly. For example, every Monday at 9:00 a.m. a report is generated and sent via email. Or every hour, n8n checks for new tweets and saves them. Think of this like an
alarm clock. You set it once and it automatically rings at the right time. Now, let us discuss about the workflow components, which are like the brains of the system. So, every n8n workflow is made up of
different type of nodes, and each node has a special role, just like team members working on a project. So, whenever we trigger the node or start the workflow, so what happens exactly? First, there is a webhook node, which it
starts someone wins, like for example, someone is sending a data to your API. starts at a specific time, like every morning at around 8:00 a.m. And then there is a Typeform trigger node, which it starts when someone fills out a
Typeform survey. So, we'll try to cover all these things in this today's tutorial. Now, there are some also nodes, like we call them as action nodes, which basically perform task. So, these nodes do not actually work in your
workflow. For example, let's say send an email node or send an email via Gmail or Outlook. Or there is an HTTP request node, which fetches a data from an API, like a weather update or something. Then there is a Google Sheet node, which like
adds a new data to a spreadsheet. So, these are called as action nodes, which basically performs task. Then you could also, you know, understand a bit about, let's say, logic nodes, which adds intelligence. So, these nodes allow you
to add conditions and decision in your workflow. For example, let's say if node checks a condition is met. Like for example, is the email from a VIP You can imagine something like this. Then there is a switch node which routes
the workflow based on different cases. Like for example, the payment example is the best one. Like if the payment is failed, send a reminder. Otherwise, if it's successful, send a confirmation. And then you could also
find out the loop node which repeats an action multiple times like sending a follow-up email every 3 days. So guys, these were some of the logics nodes which basically adds intelligence. And finally, we have the data nodes
which store and retrieves the information. These nodes basically help like Google Sheet node which helps you spreadsheet or MySQL node which stores customer data in a database or Airtable
node which basically updates information in a table. Think of it like filing cabinet. It organizes and stores important documents or data for future use. So putting up all together, let's say you run an online store and you want
to notify your team on Slack when a new order is placed in Shopify. So here is how your n8n workflow would look like. There is a trigger node like Shopify will trigger like it is going to detect a new order. Then the action node is
going to come up which is going to fetch order details from Shopify. Then there is a logic node. If the order is above 1,000 US dollars, it marks it as a high priority. An action node which basically sends a Slack message to your sales
team. So this is like the entire process which runs automatically saving your time and effort. Now guys, you would be wondering why should you use n8n? See guys, there is no need for coding skills. You can just drag and drop
nodes. It saves a lot of time by automating repetitive task. And it works with hundred of apps and APIs like Google, Slack, Shopify, ChatGPT, etc. It also has self-hosting option which ensures privacy and full control. And
finally, it is free and open source unlike some other tools which have some strict limits. Now, we are going to move forward and do a small hands-on to make you more understand about this platform. So guys, now let us create an automation
using n8n. So, first click on the first step and let's say we want to use a trigger, let's say on a webhook call. So basically, when we will be sending any HTTP request like get, post, put, or delete, it is going to trigger our
webhook on n8n. So, just click on this and you could see our method is, let's and you could see our method is, let's say, post and we want to post a data on say, post and we want to post a data on using this post request. And after
selecting this, guys, just click for listen for test event. Let's test our webhook API. Copy this and then you can go to this online tool which is called as reqbin, okay? And it is just very simple tool
where you could just do one thing. You could paste the same URL which you have copied it from here. Okay? Now, after going all from here, what are you going to do? Just click on the send request. Now, you could see we have got a
successful message that the workflow has started. But how do we sure of that this thing has working because in the output section, you will be seeing these kind of output. Okay? So, our webhook API is
working very much fine. Now, what next thing you could do? So, the next thing, let's say, let us add a node all over here. So, for example, let's say, let's add a Google Sheet, okay?
say append a row in a sheet or we want to update a row. So, now you could see all over here we have this option called append or update a this option called append or update a row from this list. And uh, let's say we
have this document called monthly expense. And you could also choose it, expense. And you could also choose it, let's say from uh, expense and we want to add uh, these properties. What values to send? Suppose name, we could set the
Neil and all over here, that's one thing. Okay? Then uh, what we could do, we could uh, add another option, like what would be
the cell format? And we could add the next property, let's say as email. next property, let's say as email. Email, let's say
Okay? Now, we could test these step. Okay? Now, you could also say let's add a column, okay, to name, okay? And uh, let's say we could add just one value, then we could add another column,
value, then we could add another column, let's say email, okay? Let's uh, another column. Okay? So, we had only two columns that I have actually created
all over here, name and email. So, that's why our workflow is showing the same. And also guys, there is one point to note, you have to also connect to your given applications, okay? So, I'll just show you how you could do it. But
okay? So, column manually. And let's test the given step.
given step. Let's wait for the output. And if I open uh, Google Sheet, you could see we can see you know uh, the name email and it's already mentioned here. So this is like how you are basically creating your
workflow. So if you go back to your canvas you could see our workflow has been created using the web a web hook call. Similarly you can trigger a lot of ways. Suppose you could add our next node like you could add AI and and all
those things and you can create this automation using any tool. Guys this is a very amazing tool and this automation tool is definitely going to save a lot &gt;&gt; So we will be discussing about what
exactly is LangChain, LangGraph, LangFlow and LangSmith and where can we use them and what makes them different from others. So this is the major agenda for today's session. Let's begin with LangChain.
So what exactly is LangChain? LangChain is an open source framework designed to help developers build applications powered by large language models by providing tools and abstractions to simplify the process of
integrating LLMs with external data sources and workflows. Now imagine you're building an AI application where an LLM model like GPT-4 is needed to query. Llama 3 or other versions to refine that
response. A custom-built AI agent to decide whether it can answer the query without a hiccup or if it has to fetch external sources to generate a response. Most importantly a solid memory to store all previous user interactions. Without
LangChain managing all these components manually would require extensive coding. You need to write a custom code snippet to handle API calls to your LLM like GPT, Llama, create and maintain a dedicated memory space, and most
importantly, a fully customized decision-making capable AI agent. Yet, opportunity to scale it up, the possibilities are questionable. This
approach leads to a lot of boilerplate code and makes the system complex. LangChain simplifies this by providing an open-source framework that helps developers build applications powered by large language models. It enables
training prompts, interacting with external data, and adding memory for contextual understanding. Now, let's discuss some of the key features of LangChain. Firstly, LLM support. It works with both closed-source GPT-4 and
open-source models like Llama 3 with simple API integrations. Prompt templates. It allows dynamic prompt customization instead of hardcoding Third one is chains. It helps connect multiple tasks like querying an LLM,
retrieve data, and processing those responses in a seamless workflow. Fourth one is indexes. Supports document loaders and vector databases to pull in external data. First one is memory. It enables applications to retain past
interactions, improving contextual awareness. And lastly, the agents. It uses LLMs as reasoning engines to dynamically decide the next step in the workflow. With LangChain, developers can efficiently build AI-powered
applications without reinventing the wheel. You can have a detailed briefing on LangChain on the official website. Also, should you wish to try it out, take a quick look at quick start page to steps to follow to set up the
environment and run LangChain in your local system. Now, let's proceed with the next one in this tutorial, which is LangGraph. So, what exactly is LangGraph? By definition, LangGraph is a library
within the LangChain's ecosystem. It is a framework for building and managing complex stateful multi-agent LLM applications by modeling workflows as graphs, where nodes represent tasks or states, and edges represent a transition
between them. Now, let us imagine this. You are in a situation where you need to work with not one, but multiple agents in parallel. But, LangChain can just assist you with one. You need much more
enhanced platform for this. That's where LangGraph comes into the picture. LangGraph is built upon the existing LangChain and designed for managing agents and multi-agent workflows. While LangChain is excellent for chaining
prompts, LangGraph excels at handling structured interactions between multiple agents. If you are building a system where multiple agents collaborate, such as a research assistant or an automation workflow, LangGraph is a great choice.
So, let's discuss these core components of LangGraph. First one, state. A state is a shared data structure that stores user inputs, agent outcomes, and workflow's history. Now, the next one is nodes. You can have one node or also
multiple nodes. These nodes are individual components or actions that execute LLM calls, run functions, or interact with external tools. Now, let's Edges are nothing but connections between nodes that define execution
flow. Unlike traditional directed graphs, nodes in LangGraph can dynamically decide which node to call next. That means the control flow can go in both ways. LangGraph is ideal when you need agents to interact in cyclical
workflows or collaborate on complex tasks. You can have a detailed briefing on LangGraph on the official website. Also, should If wish to try it out, take a quick look at the quick start. Using LangGraph is straightforward. You just
need to install the package, import it onto your project, and then using the provided classes, you can straightly access LangGraph. Now, let's proceed to the next one, which is LangFlow.
Now, what is LangFlow? LangFlow is a visual, low-code framework built upon top of LangChain, enabling developers to easily design and test AI applications, especially those involving agent systems and retrieval augmented generation, or
RAG. Now, imagine building AI-powered applications like chatbots or data processing tools without writing any code. LangFlow makes this possible with a drag-and-drop interface. LangFlow is built upon top of LangChain and provides
a visual interface for designing LangChain workflows. It is perfect for prototyping LLM applications, allowing users to quickly design workflows, chains, and agents, and test them. However, it is not intended for
production use, but it is excellent for MVPs, which is minimum viable products. LangFlow is designed to simplify AI-powered app development by providing a drag-and-drop interface. It enables developers to visually design,
experiment with, and test LangChain-based workflows. Now, what makes LangFlow different from LangChain and LangGraph? Why use LangFlow? So, there are three major reasons. Firstly, no coding required.
Perfect for prototyping AI applications without having to write any code. Second one, visual workflow builder. You can easily design chains, agents, and integrations. Thirdly, quick prototyping. Best suited for MVPs,
experimentation, and non-production use cases. So, there are two main ways to use LangFlow. First one, via DataStax, a paid cloud-hosted option. Second one, self-hosted. Install locally or on your own cloud server.
So, once deployed, LangFlow provides a UI where you can drag and connect various components enabling you to build and test AI workflows seamlessly. You can have a detailed briefing on LangFlow on the official website. Also,
should you wish to try it out, take a quick look at quick start to work on environment. You just need to install the package, import it onto your classes. Now, let's move on to the last one,
which is LangSmith. So, what exactly is LangSmith? So, LangSmith is a platform developed by LangChain designed to streamline the development, testing, debugging, and monitoring of LLM applications, offering
tools for observability, evaluation, and prompt engineering. The LLM GenAI AI agents, all the hype was a huge hit. Everybody is interested in developing a custom AI agent to automate and optimize all the redundant
CRM tasks, and much more. But, building an LLM-based application or AI agent is just the beginning. Monitoring, testing, and optimizing are equally crucial.
LangSmith is designed to help developers at every stage of an AI application's life cycle, from prototyping to production. Now, why use LangSmith? So, there are three important reasons. Firstly, performance monitoring. It
tracks API calls, token usage, errors, and latency. Second one, debugging and evaluation. It ensures models return expected results and avoid unnecessary API costs. Lastly, independent framework. It works with any LLM
framework, not just the LangChain-based ones. If your application requires extensive monitoring, debugging, or optimization, LangSmith is a great tool. However, for simple applications without complex tracking means it might not be
necessary. To integrate LangChain, install the package, set up the environment variables, and use traceable annotations in your code. The LangSmith dashboard provides insights into performance metrics, including token
usage, call trends, and error rates. For more details on LangSmith, visit the official page of LangSmith. Also, should you wish to try it out, take a look at the quick start to work on setting up LangSmith in your local
the package, import it into your project, and start using the provided &gt;&gt; Something big is happening in the world of AI, and it's shaking up the industry. No, it's not another ChatGPT update or a new release from OpenAI. This time, it's
a brand new player on the scene, DeepSeekR1, an AI model that's already outperforming the best-known models in ways that have experts seriously being talking. But why is this causing such a stir? Well, it's because DeepSeekR1
doesn't just answer questions like other AIs. It thinks. And I mean really thinks. It breaks down complex problems step-by-step, showing you exactly how it arrives at the solution. You just don't get an answer, you get the entire
process from start to finish, that too in real time. This is the kind of AI that doesn't just solve problems, it solves the way we do. Now, here's the twist. DeepSeekR1 isn't from one of the usual AI giants like OpenAI or
Anthropic. It was developed by High Fly Capital Management, a hedge fund that's quietly been making waves in the AI world. In fact, their new DeepSeekR1 light preview is already making headlines for its stunning performance
on tasks like mathematical reasoning, logical inference, and real-time decision-making, areas where even the best of the OpenAI's models often fall short. But why is this model such a game-changer? It's because in 2024, the
world is demanding AIs that can think logically and reason through problems. DeepSeek's ability to show its work is what sets it apart. While other AIs give you answers, DeepSeek lets you watch its thought
process unfold. Think of it as the AI that doesn't just solve problems, but teaches you how to think like it does. And if you think that's all just a hype, let me tell you, Forbes isn't the only one buzzing about it. Even experts
across the industry are pointing out that DeepSeek is outperforming models from heavyweights like OpenAI and Anthropic on test that measures complex reasoning. It's already winning in some major AI benchmarks, and it's only going
to get better. So, why is DeepSeek-R1 such a big deal right now? And what does it mean for the future of AI in 2024 and beyond? Well, in this video, we will explore DeepSeek-R1 starting with its unique
capabilities and how it stands out from the other AI models. We'll also test its math reasoning and problem-solving skills, comparing it with models like OpenAI. Then, we will examine the real-world application in fields like
So, let's dive in and see how it's shaping the future of AI. So, at first, let us understand what is exactly DeepSeek-R1 like to do. As you can see this tweet post, so DeepSeek-R1 like to do is an AI tool which is similar to
ChatGPT, and it's created by a Chinese company which is known as Deep Seek. The company announced this new model on X on November 20th and shared a few details. So, I'll show you the documentation page of here. So, this is a documentation
page of Deep Seek API, and DeepSeek-R1 like to do is meant to be really good at solving problem reasoning in maths, coding, and logic. It shows you how to think step by step so that you can understand how it comes up with answers
and which help people people trust more. So, you can try out me on the website you. So, you just have to simply go to Google
and type over here chat.deepseek.com. redirected to DeepSeek's page. So, simply me, I'll just log in with the
Google account. And here you can see this is the interface of our DeepSeek our light model. And here we will enable this DeepThink preview, which is a new feature over here. And now let's start with our test and start
testing whether DeepSeek this model is really that effective or worth the hype or not. So, first we'll be doing the strawberry test, okay? So, let's see what exactly is the strawberry test. So, let's start
with a very simple question, which is how many times does the letter R occur in the word strawberry? So, this is our question over here and let us just check So, I didn't expect such a long
reasoning process to what seems like a straightforward task. I thought that after counting the letter R and identifying the position of the word, it would have stopped there. But what's interesting to me is that it didn't
stop. It double-checked the counting a couple of times and even considered things like how people might pronounce or spell the word differently, which I think is a bit of pedantic, especially the pronunciation part. But then that's
how it does show how careful and thoughtful it is. So, it just explained us every step-by-step and what process we can go through it. So, let's move on to our second question, which would be a mathematical reasoning question. So, our
next question would be if a triangle So, if a triangle has sides of 3, 4, 5, what would be its area? So, let's just check. So, as you can see our Deep Seek our
model is getting our answers. And it performs the checks I predicted, although in a different order. But then, it's using all the steps and all the formulas and what would be the best to go with this question. So, both
the explanation and the output are particularly clear and easy to follow, which makes me think that this would be a fantastic model to embed in math students assistant. For example, this is a particular use case. Maybe the thought
process could be shown first and the student could interact with acknowledging whether they understood it or not. So, that's a pretty fast answer. Let's move on to our third
question. So, now let's ask our Deep Seek model a geometrical type of question, which is a bit tougher and let's see what answer would it generate for us. And how it is it taking the logic
building and the process. So, as you can see it's taking all the formulas and also it is considering the other uh probabilities as well. So, we'll just wait a few seconds.
So, as you can see it has written later messed up in a denominator. So, it is even checking what mistakes has it generated, which is really good. It is just like as if somebody is solving this mathematical problem.
But then in the form of AI as well. That's really interesting. So, now as you can see it has given us the final answer and also step-by-step approach, which is well-known geometric formulas and applied them directly,
which makes the reasoning very easy to follow. So, the output is very clear and examples, it's been impressive to see how it consistently double-check
calculations using different methods. The thought process is always deeply logical and very easy to understand. So, for somebody who wants to actually learn problems, this model will give a very detailed
explanation. So, I would highly suggest using this. So, let's now move on to our next question, which which would be a coding test. So, the first question would be uh implement a function in Python that finds the longest
palindromic substring in a given particular string. The function should particular string. The function should have a time complexity better than O n ^ 3. Big O of n ^ 3. So, let us just see what and what is the time complexity
will it generate the answer for us? The code, let's just check it out. So, as you can see, it has given us the solution code here, and also the explanation part. And whatever the function it is using,
it has briefly mentioned the explanation about it. And I think that the model did a very great job solving the problem and finding the longest palindromic substring. The approach was smart,
efficient, and clearly explained. So, instead of just brute force method every possible substring, which would be slow, it used a very clever expand around center technique. And this method handled both odd-length palindromes and
even-length palindromes as well, which was really clear and effective. So, that was for the coding part. So, now let's move on to a logical reasoning type of question, and let's see what answer does it generate. And uh
and we will also compare it with the OpenAI model here as well. So, let's quickly check it out. Our question here is a man has to cross a river with a wolf, a goat, and a cabbage. His boat can only carry himself and one other
thing. If left alone, the wolf the wolf would eat the goat and the goat would eat the cabbage. How can he get everything across safely? So, let's quickly check out the answer. So, as you can see, it's thinking of the end of
reading the answer for us. Also, for the Deep Think messages, you Also, for the Deep Think messages, you can use 50 messages free per day. So, we have already used five texts and we have 45 messages
left for today. So, as you can see, it has given a clear, detailed explanation. has given a clear, detailed explanation. And it solved the problem really very well. I think it carefully went through the rules and checked different
possibilities. It understands that some pairs like the wolf and the goat or the goat the cabbage can't be left alone. It also reviewed the constraints at the starting. It also reviews the constraints at the start.
Starting from this, it looks at what would happen if the man takes each item across the river first and works out if it creates any problem. What I find really nice is about how the model adjusts its plan when something doesn't
work. For example, when it tries taking the wolf first, it realizes talking to the wolf first but realizes at the same time that it causes trouble and then rethinks the step. This trial and error method looks very similar to how we as
humans might solve the puzzle ourselves. So, in the end, the model comes up with a very right solution and explains it in a very clear and step-by-step way. So, as you can see, I've given ChatGPT this question and this is the answer it has
generated. But then, it wasn't that detailed as compared to Deep Think detailed as compared to Deep Think model. So, here I would rate Deep Think model more because it gave me a very clear explanation with all the details
that you can clearly see the difference out here. So, for anyone who wants to understand from depth, then this Deep Seek model would be a great help. So, let's quickly have a look on the Deep Seek our own light review benchmarks.
So, we have tested the model in real world conditions, but how does it stack up against other AIs on benchmarks like AI and math and competitive programming languages. So, you have this on the screen. We have compared it with AI and
math, GPT, and diamond, code forces, light code bench, and also zebra logic. So, according to the benchmarks, Deep Seek our own light review blows the competition out of the water when it comes to math and logic problems with a
pass rate of 52.5% on the AI and 2024 challenge. It is really ahead of Open AI's own preview and surpasses models like GPT-4 on
complex math tasks. So, with this we have come to the last part of the section and here we will be comparing that who really wins. At first, you have seen that we have compared with the Open AI model. So, let's just quickly test
some questions and just check it out that with Open AI which one performs much more better. Now, we will compare Deep Seek versus Open AI own model with some command prompts to see who really wins in different categories. So, here I
have this Deep Seek model. So, my first command for this test is a math problem and it says solve for X the quadratic equation 3x^2 + 5x - 2.
So, let's quickly see this what answer is it giving and we will be comparing it with the Open AI own model as well. So, as you can see it is giving us all the
available solutions using different methods as well. And see the first formula. It's basically giving you the full overview of all the methods in which you find easier and you can use that. And
here is our final answer. So, let's compare it with the OpenAI Oven model as well. So, here I have this question and I'll just check the answer. It's taking a bit of time, but let's wait for me. So, as you can see, it has
only used one formula, which is the quadratic formula. And here we have the answer for this thing. But what I found is it provided a correct solution, but then it lacked uh whereas our Deep Seek model, it provided
So, I think our Deep Seek wins for this math question as well. For the second comparison, I have a coding problem. So, let's check for both. Uh first, we will be checking for Deep Seek
And the coding problem is write a Python function to check if a given string is palindrome or not. And I'm waiting for the answer here. So, as you can see, Deep Seek Carbon 9 Review model has provided us a correct
solution along with a detailed breakdown of why each part of the code works as it of why each part of the code works as it is does. So, let's check for the Oven model as well. So, now as you can see for the coding task, uh the Oven model
has given us the solution, but then it provides a working solution with clear code, but it doesn't explain the logic behind it. Whereas the Deep Seek model has provided the logic and also the solution with each detailed breakdown.
would be Deep Seek Carbon 9 Review because it provides better and clearer explanation. Now, let's move on to the third test, which would be the logical third test, which would be the logical reasoning. So, here is my question for
logical reasoning, which is if a man has five apples and gives three away, how many apples does he left with? So, let's check for the Deep Seek model. So, now you can see it has given us the answer and also the correct answer, but it has
explained each and every logic to go through this question and why have we used this particular logic as well. So, that's a pretty uh breakdown detail step-by-step explanation. Let's check for the OpenAI Open model as
well. So, here I have my question and let's wait for the answer. See, it has given us a straightforward answer, which is approx to apples, but then it hasn't mentioned the logic behind it, same as it was saying for the coding task. I
think uh since both of these models have provided the accurate correct answer, but here also the winner would be Deep Seek because it has provided with a lot of explanation and transparency as well. So,
this is the wrap-up of the video today and if you're looking for an AI that excels in reasoning, problem-solving and transparency, it's definitely a contender. While OpenAI and other models are fantastic, Deep Seek brings
something new to the table. Step-by-step logic, clear explanation with an edge challenges. &gt;&gt; Hey everyone, welcome to Simply Learn's YouTube channel. In this session, we will learn how to run Deep Seek locally
in Windows command prompt. But before that, let's take a look at the Deep Seek and all the hype about it. Deep Seek, the latest in generative AI, is making waves with its advanced reasoning and problem-solving capabilities. With
problem-solving capabilities. With global AI adoption accelerating, Deep Seek emerges as a powerful contender, seemingly outpacing its rivals in efficiency and depth. Built on cutting-edge architecture, it excels in
complex reasoning, coding, and multilingual understanding. AI is the new genius, and Deep Seek is pushing its boundaries further with unmatched performance. Its ability to handle intricate prompts with precision makes
it a game-changer. Unlike traditional models, Deep Seek leverages superior contextual awareness for human-like responses. Its efficiency in real-world applications refines productivity and automation. Businesses and developers
are rapidly integrating it for high-level decision-making. That's it. If these are the type of videos you'd like to watch, then hit that like and get notified. Now, before we get started, we need to check the space
left. Let me explore the file explorer once. So, you can see the downloads is almost full. Now, we have another volume D, which has a little bit of space. Let's use that. Now, let's change the location of downloads to drive D. For
that, go to downloads, open properties, and here go to location. Maybe if you want to create a new file, that's also okay. Let's try new file, that's also okay. Let's try that. So, here, let's create a file,
and I'll name this as temporary downloads. avoid confusion. And this might ask for some permissions. an application, something like that, and you set environment for that, right? So,
let's see that's the one. So, automatically, the computer will change Just in case if you have any other applications running from downloads directly. Meanwhile, while these particular files are copying, let's wait
for a moment. Let's minimize everything and open a new tab on Google. So, to run Deep Seek locally, we might want to use a few things. Firstly, Ollama. So, let's download Ollama.
Go to the Ollama thing in the third link. That's the one we need. Uh we need to download Ollama into our local system. For that, let's explore the models. So, this is the first one,
looking for. So, before downloading that, let's enable Ollama. For that, we need to download and install Ollama. Download for Windows. So, the downloading and installation might take
it. And we have started the download. Now, we will explore the DeepSeek before uh you know, all this download thing settles down. Let's explore a little bit about the DeepSeek models. So, here you
about the DeepSeek models. So, here you can see we have 7B, 1.5B, 8B, 14B. And let's also view all. So, the B stands for billion parameters. So, DeepSeek has for billion parameters. So, DeepSeek has a lot of models based on the learning
parameters, right? A couple of them have 8 billion. 671 is the highest, I guess, 8 billion. 671 is the highest, I guess, 400 GB. And uh because of space constraints, we will go with the first one, which is 1.5 billion parameters
one. You can choose the model based on your space in your computer. So, I'll go with the 1.5B one. So, we'll just click there and you'll get all the drop-down options. Go with the one you want. And so, that, just copy that code there.
Ollama run DeepSeek-R1 {double colon} one .5B, right? That's the one. And let's open the command prompt. Okay, before that, we need to install Ollama first. So, the file has been
downloaded successfully. Navigate to the downloads location. Start the setup. It setup is completely done, then we will open the command prompt. And then, copy-paste that particular code. Then, everything will be automatically
downloaded and loaded for you. So, DeepSeek will be ready to run locally on So, what's the benefit of running DeepSeek locally? Basically, when you want to communicate with your generative AI, you might want the internet
connection with you. If you are able to download and set the generative AI, in our case, DeepSeek, in your computer, then it is beneficial for you so that it then it is beneficial for you so that it can run without the internet connection.
It might take a little time. Don't worry. Once the setup is done, look at the bottom right corner. You'll get a slight notification, a smaller notification which says Ollama is set up and running,
similar to that. You can just get Ollama is running now, something like that. Just a couple of moments while the final settings are done.
Close it. Now, check the tray there. Should be having an Ollama logo present right there. There you go. That's the is running. Now, to check, you can also check the
logs of Ollama which are presently in your computer. Just for cross-verification. Also, make sure that to copy that Ollama run DeepSeek R1 code there to run it on your command prompt. Just press Windows icon
with R to have the run command, and you can type CMD to execute the command There you go. Now, make sure to copy that code. Once the Windows command prompt is active, you just have to paste it and
hit enter. Command prompt is loading. It might take a little glitch. Let's wait until it shows up. Let's try again. CMD, hit okay. Let's
prompt pops in. activities. There you go. We have multiple prompts. Let's close all the other ones which we don't need. I think it's still loading this close
this as well. Let's give it a retry. CMD okay and let's meet. So here is the we were looking at earlier. Let's close the log file which we don't need it. Yeah, the command prompt is active paste enter.
Now you should be getting something called manifesting. So that means the manifest. There you go. So based on the model that we chose which is 1.5 billion parameters it's
typically a smaller version of the DeepSeek that we were looking at. The biggest one was 675 billion parameters which is close to 400 GB. If you have enough space in your PC then you can go with the highest version. So due to
smaller one. The procedure is same. You just have to The procedure is same. You just have to paste the code and run. So at the last where you have 1.5 B right in place of that you'll be having 675 B. That's the
only difference. It might take a little bit of time to download you're going with the higher package a bigger package then the download time bigger package then the download time might even end up more than 24 hours.
And if you don't have enough space it might collapse in between. So I I'd select the one which suits with your memory space. Now the files are downloading. It might take a little time. Let's wait for that.
third. So I think it's downloading in terms of packages. So we have the fifth package. Verifying. There you go. Successfully downloaded. Now
we can type in a message. Let's say let's type hi DeepSeek or hello DeepSeek. Verify. Verify. So hi DeepSeek. So you can see when you
thinking mode right. You can see it is also thinking. Now, let me type my request here. So, it also said as "Hello, how can I assist you?" So, my request is I wish to learn neural networks. So, you can see that when you
open Deep Seek on Windows browser or Google browser, Firefox, any of the browsers, it'll show you the thinking process, right? Here also you can see that thinking process, start of thinking and end of think. The only thing is
unable to display the thought process over here. But, fine. Now, let's continue with the command prompt, which is I wish to learn neural networks and is I wish to learn neural networks and hit enter. Now, let's say it will help
you to learn neural networks in a stage-by-stage process in the learning order. What exactly is neural networks, what are the key components of neural networks, neurons, activation, weights, and biases, and everything, right?
Similar to ChatGPT or even other competitors in terms of generative AI. And it is also giving you procedures and sample codes here, right? If you wanted to execute these codes on Jupyter notebook, then you can just import NumPy
as NP, from TensorFlow, etc. And also you have some sample codes for generating a model, how to execute that, everything. So, I guess that's the end of the line for the prompt.
Now, this was the trial run on how to run Deep Seek locally on your Windows command prompt. Now, you can also develop a completely new user interface and experience for your Deep Seek to run on locally. So far, so good. For the
sample purpose, this is good. Now, let's type in "Bye" and see how it responds to the text. There you go. It might be confused, but it might give you an alternate approach. So, it looks like your message got cut
off. Okay, could you clarify? Today we are exploring Hugging Face, an amazing tool that makes working with language and AI super easy. If you're curious about using advanced technology to understand or create text, this video is
perfect for you. Hugging Face is a company that helps people use AI models for language tasks like translation, text analysis, or even generating new text. They have created a library called Transformer, which comes with
pre-trained models, so you don't have to build everything from scratch. It's simple, powerful, and perfect for developers, researchers, or even beginners. So, in this video, I will show you three cool things you can do
with Hugging Face. Speech to text, turn spoken words into written text easily, great for captions or voice apps. Second one is sentiment analysis. Find out if a text is positive, negative, or neutral. Helpful for understanding reviews or
comments. The third one is text generation. Create human-like text that sounds natural. Perfect for chatbots or creative writing. I will also explain some important basics, like how pipelines make using Hugging Face models
super easy, and how tokenization helps AI understand text. By end of this video, you will know how to use Hugging Face to start building your own projects. It's simple, fun, and really powerful. So, without any further ado,
let's get started. So, welcome to this demo part of this video. three things. First thing, speech to text recognition. The second thing, text generation from a particular sentence or
a word. And the third thing we will do sentiment analysis. Okay? So, we will perform this one by one. So, let's start with something cool, speech to text recognition. Okay? Using Hugging Face.
recognition. Okay? Using Hugging Face. So, first I will write rename this. So, first I will write rename this. I'll write here Hugging Face I'll write here Hugging Face speech to
Okay. Yeah. So, first I will install Yeah. So, first I will install transformers library. It is already installed on my system, but again,
just for you guys, I'm doing this. Okay. PIP install transformer. So, now let's see what is transformer. So, transformers is a powerful Python library created by Hugging Face. So, that allows you to download,
manipulate, and run thousands of pre-trained open-source AI model. So, as you can see, requirement already satisfied. It means like already installed. So, these transformer models cover multiple tasks across, you know,
modalities like NLP, natural language processing, computer vision, audio, and multimodal learning. Like many things. Fine. Fine. So, now I will do from transformers
pipeline. Okay. So, now let's run it. So, now what is pipeline? Okay. So, a transformer pipeline describes the
flow of data from origin system to destination system. Fine. Because you already know transformer means you can run or manipulate thousands of pre-trained open-source AI
So, pipeline describes the flow of data from origin system to destination system and defines how to transform the data along the way. Okay. So, let's check the
versions. Transform Transform versions.
That long time. Okay, why it is coming? Import transforms. formers formers Yeah. So, we have currently 4.42.4
version of the transformer. Okay, so I am using Google Colab. Okay, so you can use either Jupyter Notebook, Visual Studio Studio or Colab.
Okay. So now, let's import libraries. So now, let's import libraries. Import librosa. So, what is librosa? So, librosa is a Python package for
audio and music analysis. Right? Because we are doing speech-to-text, so we need this. So, it provides various functions to quickly provides various functions to quickly extract key audio features and metrics
from the audio files. Okay, and this librosa can also be used to analyze and manipulate audio files in a variety of formats such as a variety of formats such as WAV, MP3, M4A, and like that.
Okay. So next, we will import torch. I hope everyone know torch. torch is nothing PyTorch. It is a machine learning library based
on the torch library used for applications such as computer vision and NLP. And originally, it was developed by the Meta AI and now the part of Linux Foundation, you know, umbrella.
Foundation, you know, umbrella. Let's import one more thing. Import Let's import one more thing. Import So now, let's import
Capital. IPython. dot So, now you will be wondering what is this IPython display.
So, it is an interactive command line for the terminal for Python. And it you know, it can provide a IPython terminal and the web-based notebook platform for Python computing.
And this IPython have you know, more advanced features than the Python standard interpreter and we can quickly execute a single line of Python code. Nothing. Okay, for that I'm using this.
Okay, for that I'm using this. And the next is from transformers And the next is from transformers import WAV2 WAV2
And WAV2 WAV2 WAV2 WAV2 WAV2 WAV2 Tokenizer.
the notebook on how to fine-tune a speech recognition model. Okay. And this tokenizer, so tokenizer is nothing Tokenization is a conversion
of a text into meaningful lexical tokens belonging to categories defined belonging to categories defined a lexical program. In case of NLP, so those categories include nouns, verbs, adjectives, punctuation, etc. Okay, now
adjectives, punctuation, etc. Okay, now and the last library let's import numpy and the last library let's import numpy as np. I hope everyone know numpy, what library used for working with arrays. It also
It also has a function, you know, for working in domain like linear algebra, Fourier transform, and matrices, and many others. Fine. I hope everyone know about these library, which we have
Okay, so let's move forward by tokenizer.
Google do Google me Google do Google do tokenizer.from
just write just some words and it will show the suggestions. You know, Facebook So, this is again one pre-trained model, okay?
model, okay? So, we are just importing it. This is nothing, the name. Okay, then model equals to
for CTC .from pre-trained then Facebook
vectorizer do do then base
Now, let's run it. So, now you can see Okay, they are they are downloading.
So, now let's load the audio file. So, here I will write audio So, here I will write audio on my sampling
librosa dot dot load So, you know about librosa, right? Then
I have v. m4a So, we I have already one s- speech or one audio, you can say. speech or one audio, you can say. So,
the final output, I will you know 16,000 Okay.
Okay. Already loaded, I guess. Okay. Okay, no issues. I will load again.
File is loaded. I will rename it to v. Yeah. Fine. So, now it won't the error won't come.
We have v.m4a. Okay. So, now we have to do
Let me try it. We have v.mp4a, We have v.mp4a, but I don't know why it is not coming.
Okay, let me copy the path. Now it will run, I guess. Yeah. So now I will write here audio
{comma} sampling rate. Okay. Now listen carefully. Then I will write display.
path {comma}
true. Okay. message. &gt;&gt; I guess you heard, but again let me play
&gt;&gt; Hello and welcome. This is an AI voice message. &gt;&gt; Okay. So it is saying hello and welcome. This is an AI voice message. Fine. So now what I will do, I will input some
So now what I will do, I will input some values. Input values. Input values {equals} to values {equals} to tokenizer
audio {comma} return tensors {equals} to PT.
dot input values. Okay. Then here input
Okay. Then here input values. Okay. Input values. Okay.
Yeah. So now logits will come here. So we have to now store the logits, which means non-normalized predictions. means non-normalized predictions. Okay. So logits equals to
Okay. So logits equals to model values. Dot logits. Okay. Then here I will write logits.
So now what we will do, we will store predicted IDs, then we will pass the logits to values to softmax to get the predictive value. predictive value. Okay. So here I will write predicted
IDs equals to Okay. Then logits. Comma dimension
Comma dimension equals to minus one. prediction uh to the tokenizer decode to the transcription.
Okay. So here I will write transcriptions equals to tokenizer.
Dot decode. IDs zero.
So, now let's see our output transcriptions. So, now you can see "Hello and welcome. This is an AI voice voice message." So, &gt;&gt; Hello and welcome. This is an AI voice
&gt;&gt; Amazing, right? So, this is how you can recognition. It's a very small code line of code, as you can see. Okay. So, now let's move forward and do
the sentiment analysis. Okay? Or text generation. Okay? Or text generation. So, let me open new drive. So, yeah. So, now I will write it here.
I will write sentiment {slash} text generation
Okay? So, first of all, let's import the So, first of all, let's import the files. So, first I will write import import warnings. Then warnings
dot filter warnings. Okay? Okay? I will write here ignore. NumPy. You already know what is NumPy
and import Pandas. So, these are some basic Python library and I hope everyone knows. And import Matplotlib for the plots.
Matplotlib. dot pyplot as plt. as plt. Then I will import seaborn for the
graphs. The static things. Okay. As soon as that sns.set
So now we'll import sklearn.model_selection. probably the most useful library for machine learning Python. So So this library contains a lot of efficient tools for machine learning and
you know statistical modeling including classification you can do regression you clustering you can do from this sklearn. Okay. From
dot model selection
test split from sklearn.
metrics import import f1_score comma confusion_matrix
So again we'll import things from transformers. pipeline. Everyone know what is this. Then import
torch. Okay. Now let's run it. Terminate existing to continue if you are interested in using our more session. Okay. Wait.
Fine now. Which one is open, bro? One minute this. Terminate this. Terminate this. No.
Okay. Fine. So now let's do sentiment analysis. Okay. The hugging face library provides, you
know, convenient pipeline as you already know function that allows us to easily perform sentiment on the text. So now let's first import the necessary dependencies
and create a sentiments pipeline using this line. So I will write here classifier equals to equals to pipeline
sentiment analysis. type classifier.
So it will download the pre-trained sentiment analysis pipeline. Pipeline is not defined. Okay.
Okay. Okay, error is there. Mhm. Yeah. So now we can uh pass the single sentence or a list of sentence to the classifier and now and
get the predictive sentiment labels and associate Fine. So now just for the testing classifier. So let's write this
is a great So now you can see here label positive. So label positive typically refers to
the outcome or a class of interest that the model is designed to predict. So here we are just checking the sentiment analysis model. Okay. So this is why I wrote this.
Okay. So let's check one more.
great course. I did not
understood any of it. Now let's check this. Yeah, perfect. And the score you can see the accuracy score 99% 99% which is
almost close to 100% and which is amazing. So now you have access to a GPU so you can also utilize this for the faster processing by specifying device using the parameter. Okay. So now
what I will do first I will Yeah. Fine. So, now I will import data set.
Fine. So, now I will import data set. Airline tweets. pd means pandas library here we are using.
Okay, now first import I will import here itself. here itself. So, I have twitter_tweets.csv.
.csv Let it upload. Let it upload. Yeah, done. So, now let me Yeah, done. So, now let me airline
tweets .head Why dot head? Means you it will show me the top five rows. Okay, 0 1 2 3 4 5. Okay, you can see tweet ID, airline
sentiment, neutral, positive, neutral, negative, then this is this. Okay. So, now let's do something. Okay. So, what I will do df equal df means data frame airline
tweets Then I will write airline sentiment We have airline sentiment this column. comma text
I need these two columns. Again, df.head So, airline airline sentiment is a neutral and this is what this text is
all about. Okay? Because these two are the main things, text and the sentiment. So, now let's make plot. So, now let's make plot. Count plot.
x equals to airline sentiment.
plt.y label
draw one graph. Okay? It will show neutral positive sentiment and the negative. So, as you can see the
negative sentences or the sentiments are the more. So, now So, we have now three classes which do not match the two class available in the hugging face pipeline. So, therefore, we will filter out all
the rows which have been labeled as neutral. So, df equals to df and again df airline sentiment.
Fine. was not equals to was not equals to neutral. Okay. Then DF
target equals to DF airline sentiment uh map
Then I will write here positive. Negative will be
Okay. Then print
11,541. Okay. So now I will add predictions.
Okay. Predictions is not defined. Okay. So now what I will do text equals to DF
dot to list. Then here I will write predictions equals to classifier
So, here write probabilities equals to prediction then score
score if predictions starts with
Okay. else 1 minus prediction score prediction score prediction
Okay. Then I will write for prediction and prediction prediction Why it is not running?
This is taking time, too much time. I don't know why. So, as you can see finally we have the output our predictions values. So, this depends on you know, system to system. My system took almost 17 minutes 40 47
seconds to complete. Okay. So, now let's run it. Okay. Predictions is not defined. Oh, no. predictions Yeah. So, now I will write here
predictions equals to NP NP dot array NP means numpy pi dot array dot array NP means numpy pi dot array Then I will write one if prediction
label is positive dot starts with Okay, else
Okay, else zero prediction and prediction values. and prediction values. Yep.
accuracy of our model. Print accuracy. Okay.
Then, I will do round up the values. So, here I'll write np dot the values. So, here I'll write np dot mean. then the target right.
Yeah. Then, I will write equals equals to predictions. Then, I will write into 100 comma two
two then I will write percentage. Fine, looks good. Yes. So, as you can see, our accuracy is
88.99%, which is, you know, very good again. which is, you know, very good again. So, now let's do some confusion matrix. So, now let's do some confusion matrix. Confusion
matrix then df target comma, predictions, comma, normalize
equals to true. Then I will plot I will plot confusion matrix.
normalize equals to true. Okay, okay, my bad, my bad, my bad.
labels. Okay. So here uh I will, you know, plot a confusion matrix using seaborn. Okay, so here I will use a r g s which
is confusion matrix n p dot n array. So which is, you know, labels list. So I which is, you know, labels list. So I will write p l t dot figure
equals to eight comma six. Plus s n s dot set set font scale
equals to one point four. Okay. Then let's create the heat map.
Confusion matrix. comma and not will be true. equals to T. Then confusion map
Then confusion map equals to equals to labels. Then Y
ticks labels equals to labels. Okay.
Yes, so now I will write PLT dot title will be confusion matrix. Then PLT dot X label will be
will be actual values. Then PLT dot actual values. Then PLT dot show.
Okay. Why the chart is not coming?
positive. Okay, now the chart will come.
So now you can see here this is in the actual and this is a confusion matrix and the negative and the positive ratio Okay. So now let's print the
uh now let's check the ROC AUC score. So I will write here print ROC AUC score.
Then here I will write ROC AUC score. Then DF comma probs. probs. Okay.
what is this ROC AUC score. So this is the area under the ROC curve. So what it does, it sum up how well a model can produce relative scores to discriminate between positive and the negative instance across all the classification
threshold. So with this ROC score of 94 0.94, which is 94%, we can conclude that the that our pre-trained sentiment analysis model
and effectiveness so in predicting the sentiment labels. capable of accuracy classifying text into positive or the negative sentiment
categories. Okay. So now we'll do text generation. Okay, so text generation involves generating creative and coherent thing. So I will write here text.
Okay. So, text generation involves generating creative and coherent text based on a given prompt or starting point. So, what necessary dependencies and load the data data set of the poem.
Okay. So, here I'm write poems. Let me write poems equals to pd.read .csv. Don't worry, I will give you these files. Okay.
So, here I'll I'll write Robert Frost.
And then I will write poems .head .head five. Or head, just have to write head. Module pandas has
Module pandas has Okay, not dot underscore csv. Okay, it will show the top five. Stopping by woods on a snowy evening, fire and ice, the aim was song, collection, content, year of
collection, content, year of publication. Okay. publication. Okay. So, now what we will do, we will write equals to poems
content .drop NA to to list.
So, to generate text, we extract individual lines from the poems and use the pipeline text generation function to create a text generation pipeline. Fine? So, here I will write lines equals to
Then I will write for poem in poem in content in poem dot
split into this next line. into this next line. Okay?
Then I will write line dot write a strip. To the right it will add. To the right it will add. Okay?
So, now lines equals to line for line in lines
of line then show me the lines.
Okay. So, now you see as seen here whose woods these are I think I know. Then the next line is ours is in the village though. Then the next line he will not though. Then the next line he will not sing. Okay? Like this. So, now let's
you know, import that pipeline text generation module. Okay? Gen equals to
model. You already know You already know generation.
Okay. So now I will write here lines. Yeah, done. So in line zero you have we have whose
words these are I think I know. Okay, so now we can and a specific find the parameters such as max length and num return sequence. Okay, why because we have imported this
text generation module. Okay, for example, see gen gen line Okay, sorry. lines
zero dot max length generate the this to the maximum 20.
till 20 word maximum length. Okay, check. Okay, uh Okay, gen expression cannot contain
perhaps double equals to Well, okay. Now I will run. So see, the line was this much only. Whose words these are I think I know. Whose words
But here the generated text I wish to go to church because I feel like. Okay, this is how you can do you know text generation. Now let's check for the more.
gen lines one lines one Okay, then max comma num
return sequence cells is equal to two. Okay. So, our first line was this, his house is in the village though. Okay.
0 1 2 3 4 5 Okay. So, here you can see. See, his house in the village though how well you might say that the place was the same with the This is this is this. So, these are the generated text.
Okay, and this is another you know, return sequence second. Okay. So, now So, now let me, you know, import
let me, you know, import text app. creating function wrap X X I don't need this.
dot fill X comma replace
white space is equal to false.
sentence endings is equal to true. Okay, so now out out is equal to generated lines.
Zero to maximum length. Then print
to zero. Then Okay. So, now we are setting the bad tokens to EOS tokens, okay? So,
whose whose these are I think I know. And this is the maximum 30 till 30. So, now I will write here. Okay, preview equals to Okay.
Okay. So, now what you can do, you can you know, generate a prompt to generate text on a specific topic like this. text on a specific topic like this. Prompt equals to
have a wide variety variety of applications
Okay. So, this is my prompt. Okay, this is I'm not importing from the data set. Okay. So, I will write out equals to gen
gen prompt, {comma} max length
the output out. Okay, here I will write Anyways, here I will write generated
So, it is running. Let's wait. Okay, some error is there.
words. I wrote I wrote 50. So, let me do it again. 100.
So, now you can see we have we can generate text using So, we have covered, you know, three topics First thing is speech to text using hugging face. Second thing is sentiment
And the third is text generation using hugging face. And in this text generation we have we did with two methods. One with data set
and second thing with the you know, by giving prompt like in ChatGPT, you can see. &gt;&gt; On July 25th, OpenAI introduced ChatGPT, a new search tool changing how we find information online.
Unlike traditional search engines, which require you to type in specific keywords, ChatGPT lets you ask question in natural everyday language just like having a conversation. So, this is a big shift from how we were used to searching
the web. Instead of thinking in keywords and hoping to find the right result, you can ask now Search GPT exactly what you want to know and it will understand the context and give you direct answers. It's designed to make searching easier
and more intuitive without going through links and pages. But, with this new way of searching, so there are some important questions to consider. Can Search GPT compete with Google, the search giant we all know? What makes
Search GPT different from AI overviews, another recent search tool? And how does it compare to ChatGPT, Open AI's popular conversational AI? So, in this video we are going to explore these
questions and more. We will look at what makes Search GPT special, how it compares to other tools, and why it might change the way we search for information. Whether you are new into tech or just curious, this video will
break it down in simple words. Stick around to learn more about Search GPT. So, without any further ado, let's get started. So, what is Search GPT? Search GPT is a new search engine prototype developed by Open AI designed
to enhance the way we search for information using AI. Unlike a typical chatbot like ChatGPT, Search GPT isn't just about having a conversation. It's focused on improving the search experience with some key features. The
first one is direct answer. Instead of simply showing you a list of links, Search GPT delivers direct answer to your question. For example, if you ask, cancellation headphone in 2024?" Search GPT will summarize the top choices
highlighting their pros and cons based on expert reviews and user opinions. traditional search engines that typically provide a list of links leading to various articles or videos. The second one is relevant sources.
Search GPT responses come with clear citations and links to the original sources, ensuring transparency and accuracy. So, this way you can easily verify the information and delve deeper into the topic if you want. The third
one, conversational search. Search GPT allows you to have a back-and-forth dialogue with the search engine. You can ask follow-up questions or refine your original query based on the responsive you receive, making your search
experience more interactive and personalized. Now, let's jump into the next topic, which is Search GPT versus Google. So, Search GPT is being talked about a major competitor to Google in the future. So, let's break down how
they differ in their approach to search. The first one is conversational versus keyword-based search. Search GPT uses a conversational interface, allowing user to ask question in natural language and refine their queries through follow-up
questions. So, this creates a more interactive search experience. On the keyword-based search, where a user enter pages. The second thing is direct answer versus
list of links. So, one of the Search GPT's standout feature is its ability to provide direct answers to the question. It summarizes information from the various sources and clearly cites them. So, you don't have to click through
Google typically present a list of links, leaving user to sift through the need. The third one, AI-powered understanding versus keyword matching.
Search GPT uses AI to understand the intent behind your question, offering more relevant result even if your query isn't perfectly worded. Google's primary method is keyword matching, which can sometimes lead to less accurate result,
especially for complex queries. The fourth one, dynamic context versus isolated searches. So, Search GPT maintains content across multiple interaction, allowing for more personalized responses.
Whereas Google treats each search as a separate query without remembering previous interaction. And the last one, real-time information versus index web pages. Search GPT's aim to provide the latest information using real-time data
from the web. Whereas Google vast index is comprehensive but may include outdated or less relevant information. So, now let's jump into the next topic, which is Search GPT versus AI overviews. So, Search GPT and AI overviews both use
AI, but they approach search and information delivery differently. It's also worth noting that both tools are still being developed, so their features and capabilities may evolve and even overlap as they grow. So, here are the
differences. The first one is source attribution. Search GPT provides clear and direct citation link to the original sources, making it easy for user to verify the information. Whereas AI overviews include links, the citation
may not always be clear or directly associated with specific claims. The second one is transparency control. Search GPT promises greater transparency by offering publishers control over how their content is used, including the
option to opt out of AI training. AI overviews offer less transparency regarding the selection of content and the summarization process used. The next one is scope and depth. Search GPT strives to deliver detailed
and comprehensive answers pulling from a broad range of sources, including potential multimedia content. And in AI overviews offers a concise summary of key points, often with links for further exploration but with a more limited
scope. Now, let's jump into the next part. Search GPT versus ChatGPT. Search GPT and ChatGPT, both developed by OpenAI, share some core features but serve different purposes. So, here are some
differences. The first one is primary purpose. Search GPT designed for search, the web. Whereas Chat GPT focus on conversational AI, generating text responses. The second one is information sources.
Search GPT relies on real-time information from the web. Whereas Chat data, which might not be current. The third one is response format. Search GPT prioritize concise answers with citation and source links. So,
generating longer text, summarizes, creative content, code, and etc. The next feature is use cases. Search GPT ideal for fact-finding, research, and task requiring up-to-date information. Whereas Chat GPT is
suitable for creative writing, brainstorming, drafting emails, and other open-ended tasks. So, now question arises, when will Search GPT is currently in a limited
prototype phase, meaning it's not yet widely available. Open AI is testing and improve the tool. So, if you are interested in trying Search GPT, so you can join the waitlist on its web page, but you will need a
Chat GPT account. A full public release by the end of 2024 is unlikely as Open AI hasn't set a timeline. It's more probable that Search GPT features will gradually added to the Chat GPT in 2024 or in 2025 with a potential standalone
release later based on testing and the feedback. So, with this we have come to question or doubt, please feel free to ask in the comment section below. Our possible. &gt;&gt; Let's start with a simple question. Why
should we use LangChain? Imagine you're working with large language models like GPT-4 or Hugging Face models, and you want to take their capabilities further, like integrating them with your own data sources or allowing them to take action
based on information they retrieve. This is where LangChain comes in. LangChain allows you to build intelligent applications by connecting large sources. It can turn static AI interactions into
dynamic data-aware workflows. One of the best parts is you don't have to manually code everything from scratch. LangChain abstracts away much more complexity of working with LLMs, allowing developers to focus on building functional
applications instead of wrangling API calls and managing data pipelines. So, LangChain is set to play even bigger role in AI development because it enables you to harness true power of generative AI by connecting it with
real-time data and external tools. So, now we have understood what LangChain is. Let us now understand how to install LangChain. So, let's start with the installation of LangChain. Uh we'll just simply go to the website
and we'll just simply go to the website docs part and we'll just read through this documentation. So, here it has explained what LangChain is and what are
the framework consisting of. So, we also have these tutorials on how do we install LangChain. Okay. So, for installing you can just simply click on this quick start and uh see here it has written how do we set up on Jupyter
notebook. So, this is the command if you want to install LangChain. We will use the PIP command. So, just simply you can copy this command PIP install LangChain and you have to open your command prompt or the terminal in your computer and
here you have to simply copy-paste the command. So, as you can see it has it will load all the packages which is required for installing LangChain. So, you can see here requirement already satisfied. This is because I had already
satisfied. This is because I had already installed my LangChain before. So, uh we have understood how to install this LangChain by using this command and you can also install the LM chain. We'll understand it later. Let me just show
First, we have understood this LangChain, then we have the Pinecone client. So, we'll just simply search here Pinecone client
and it will redirect us to this page. So, Pinecone client is actually a vector store for storing storing and retrieving embeddings, which we will use in the later steps. So, Pinecone is also used
to actually, you know, create secret API keys. So, here you can also create the API keys. You can also read the documentation part. So, so we'll understand how to create API
keys using OpenAI, but first, let us install Pinecone client in our system. So, we'll again go to command prompt this terminal and we'll just copy-paste pip install Pinecone client. So, you can see here it will download and install
all the packages required. So, it has been installed. Now, the third thing we're talking about is OpenAI client. So, we'll use OpenAI models for a So, we'll use OpenAI models for a language large language task. So,
so we'll just simply search here OpenAI, okay? And so, it has redirected us to this page, OpenAI platform. And uh Okay,
before starting this, so this is the platform here where you so this is the platform here where you can create, export an API key uh in OpenAI, okay? So, you can see here overview, quick start, concept,
everything is there. And uh To create an API key, we'll just simply click here and here you have to select this option, create a new secret key. Suppose I give my secret key name, anything you can
give. Suppose uh I give test test 1 2 3, okay? And permissions is all and we create the secret key. Now, uh you need to uh actually save your key. We'll just copy this key
because it will be required later while uh debugging the code. So, we will just copy-paste this secret key. We will require it later and then done. So, these are the keys I have created. So, actually ChatGPT and other LLM models
like OpenAI and Hugging Face uses LangChain to integrate with other APIs to create your own custom LLM models or chatbots. So, suppose here we have uh logged in our ChatGPT here and uh if I
search here who won the uh who won the uh WTC World Cup
So, here it has shown the answer, but for example, uh if I search for example, uh if I search who won the cricket who won the cricket match World Cup.
given answer as "As my last knowledge updated in April, men's World Cup has not taken place yet." This is happening because the uh this ChatGPT uh older version has not been trained on the latest uh upcoming news or whatever the
new technology is. So, by using LangChain, you can integrate with other APIs and you can create your own customized LLM models or chatbots, which help you to train your own custom data using various tool and
APIs. So, uh before we move on, I've already showed you how to create the secret uh API key and how you have to store that uh API key address. So, first we have already understood how do we install
LangChain here. So, by using the pip command we do it. And command we do it. And So, you also need to install Python 3.8 or later installed in your system. So, I already have Python installed. And to
check that, you can just simply I already have it installed in my system. here Python minus minus version. And click enter. So, as you can see it has shown me the Python version which is
installed in my computer. This So, the second step is already we have discussed which is OpenAI API key here. So, second step we have already API key. So, we have to sign up in our
OpenAI, then go to the API key section, and then create a new secret key. And these are the keys I have created, and you can just keep it later later you'll use that. So, now we'll come to the third step which is create a project
third step which is create a project directory and setup. So, what we do is we have Jupyter installed in our system. So, we'll just go to command prompt here and type j u p y Jupyter notebook.
So, it will redirect us to the Jupyter notebook installed in our system. So, if not you can just simply go from here. wait. Now, you just need to click on this
new and Python 3 I I kernel because Python has been installed in my system. here. And here you can just give the prompts the command. Before that, you have to create a Python file which we can also
create this Python file in Visual Studio Code. Just simply go to Visual Studio Code here and just simply click on file, new file
and just simply click on file, new file and I'll just type your python.py. And you can you have to first create store the API key. For this we use the open AI
underscore API underscore key equal to equal to and just give your secret key.
your secret key and just store this. So this ensures that your API key is stored securely and it can be used whenever needed. Now the step four is to initialize the project and install the required libraries. So you need to add
some additional libraries like Streamlit to make a user interface. So let's add that to in our project folder. You can either create a text file named requirements.txt and then install all these. We have already
installed the open AI LangChain. We just need to install Streamlit. So here you need to install Streamlit. So here you can just give the command pip install Streamlit. So as you can see this I have already installed Streamlit before. Same
wise you can also install open AI if it is not installed in your system using the command terminal in Windows PowerShell. So we have understood this how what all packages and what all we need to
install. Now the next step is to build your first LangChain app. To create a simple app which uses input query and the app will generate a response using open AI's GPT model. So you have to create a Python file named
main.py here. So So as you can see I've already imported this main.py and this is my code here. Import Streamlit as ST from LangChain and the constants I've created and then I have initialized the OpenAI
with our API key. So, you have to you have to just type this prompt. Here, I'm using VS Code here. You can also do it in your Jupyter notebook. And then to create the Streamlit app, you have to give a title, LangChain demo with
OpenAI. So, this is the title I have provided. And then the text input prompt for prompt, the prompt is st.text_input and enter the prompt. You can just type enter the prompt or whatever you wish to. And then display the response. So,
if prompt response is llm.predict prompt, so you can use the predict method for llm. So, here what the app will So, after creating and debugging this in the terminal, so your app will initialize OpenAI using your API key and
the user inputs a prompt through the Streamlit interface. LangChain processes the input and sends it to the OpenAI's GPT model and the AI generates a response which is then displayed in the app. So, now you can use all these
prompts to run on your app. So, to do this, you can just to see your app in and run the following command which is streamlit run main.py. So, you can just simply go to the terminal here and just simply type
uh simply type the command which is stream run main.py. your browser will open displaying the app and you can also type any question
to the input box. So, last, now we have understood all these steps. So, this was a quite basic tutorial on how to install LangChain and then, you know, integrate it with the app. You can also customize and expand. So, LangChain's flexibility
allows you to integrate other APIs also, external data sources, or even add memory to your AI application. So, whether you're building a simple chatbot or a more complex AI system, the possibilities are endless. So, by
following all these steps, you will have a fully functioning app running in your system in no time. What if I have told you I've tested over 1,000 AI tools? Yep, after all that exploring, experimenting, and a few late nights,
I've finally narrowed them down to absolutely the best. In this video, I'm absolutely the best. In this video, I'm going to share my top picks of 10 tools from each of the five biggest AI categories that are actually worth your
time. Whether you're into image creation, writing, voice tools, video generation, or AI coding assistants, I have got something lined up for you. These aren't random tools that kind of work. These are the ones that stood out,
the ones that genuinely get the job done. So, if you have ever felt overwhelmed by the flood of AI tools out there, this video is going to be your cheat sheet to the top of all commerce in every category. Let's start with the
category of image creation. In the first place, we have Midjourney. If you're after breathtaking artistic visuals, this one is for you. It's especially great for concept art, fantasy scenes, or anything with surreal polished look.
or anything with surreal polished look. Next is DALL-E 3, integrated right inside the ChatGPT. What makes this tool stand out is how well it understand prompts. It nails small details, and you can even edit parts of your image
directly. Super useful for content creators or marketers who need fast, accurate visuals. Third is Stable Diffusion, and it's go-to for anyone who wants full control. It's open source, so you can run it locally, train your own
models, and even build your own styles. If you're technically or just love to tinker, this one is for you. Coming to number four is Leonardo AI.
It's made with game developers and storytellers in mind. It has pre-trained models for characters, environments, and even items. So, if you are building a
world, this tool makes it faster and easy to bring it to life. At number five, we have Adobe Firefly. This one's perfect for professionals. It fits right into the Adobe ecosystem if you're already using the Photoshop or
Illustrator. It's a smooth experience, the image quality is high, and text effects are great for branding or designing work. Number six is Playground
AI. This is probably the most flexible tool on this list. It's web-based, fast, and great for trying out multiple styles without any complexity. If you want something lightweight but powerful, definitely check this out. Art Breeder
comes in at number seven, and it's a bit different from others. Instead of generating images from prompts, you mix and evolve existing images. Great for character design, especially faces. It's
super collaborative and creative, though a bit limited in image resolution. Number eight is Nightcafe. This one's great if you want variety. It supports multiple generation styles like VQGAN and Stable Diffusion. There is also a
strong community aspect, so you can share your work and get inspired by share your work and get inspired by others. At number nine, we have Fotor. It's clean, simple, and ideal for creating social media graphics, posters,
or marketing visuals. You don't need any design skills. It's perfect for your quick results. And finally, number 10 is deep dream generator. This one's all about the weird and abstract. It creates psychical
dream-like visuals that are super unique. It won't fit every project, but experimental or surreal work, it's awesome. Next category is voice generation. These AI tools can narrate, speak, or even clone your voice with
shockingly realism. Whether you're creating YouTube videos, podcasts, audiobooks, or building characters for games, these are the voice tools that actually deliver. In the first place, we have ElevenLabs.
It's the most natural-sounding AI voice out there. You get emotion, depth, and even voice cloning that sounds incredibly real. Coming to the second is Murf AI. This one is great for polished voiceover, perfect for videos,
presentation, and training content. It even includes background music and timing control. At number three, we have got LOVO AI, also known as Genny. With a
huge voice library and support for 100 plus languages, it's ideal for creators who want expensive, multilingual voiceovers. In the fourth place, we have play.hd. It's fast, reliable, and offers detailed
control over the tone, pitch, and speed. Great for podcast, articles, or any voice first content. Number five goes to Resemble AI. This one's all about
real-time voice cloning and emotionally delivering. Perfect for interactive content like games and virtual assistants. At number six, we have Descript Overdub. Audio by editing text. It's a lifesaver for podcasters and
video creators fixing mistakes or adding new lines. Seventh on this list is Typecast. It combines realistic voice AI with animated characters, making it perfect for explainer videos, YouTubers, and
storytelling content. At number eight, we have Speechelo. Originally made for reading, but now you can use it to turn any text into clean, human-like speech.
Great for audiobooks and productivity. Number nine is Synthesys, a solid choice for business voice over, whether it's ads, explainers, or training videos.
This one gets it done with professional sounds. And finally, in the 10th place, we have Podcastle. It's an all-in-one podcasting platform with built-in voice generation, recording, and editing. Perfect for creators on the go. Now,
let's talk about writing. Whether you're crafting blog posts, emails, scripts, or social media captions, AI writing tools can save you hours and spark ideas you
wouldn't have thought of. I have tested dozens, and these are 10 that stood out how well they actually help you write smarter and faster. In the first place, smarter and faster. In the first place, we have ChatGPT by OpenAI. It's a gold
standard. From brainstorming to full-on article drafts, it handles everything with context and style. Especially with GPT-4, it's like having a writing GPT-4, it's like having a writing assistant that just gets it. Second is
Jasper AI. Built specially for marketers and content creators, Jasper comes with tons of pre-built templates for product description, ad copy, blog intros, you
name it. It's fast, efficient, and optimized for high-converting content. At number three, we have Rytr. This one's amazing for versatility. It
does blogs, landing pages, Google Ads, and even long-form content. It also has a chatbot, so it feels more interactive while creating. Fourth is copy.ai. If you need social media post, product
If you need social media post, product description, or email subject lines in seconds, this tool nails it. It's super intuitive and beginner-friendly, perfect for quick content sprints. In the fifth place, we have Notion AI.
Built right into Notion, this tool helps you expand, summarize, and brainstorm inside your notes and docs. Great for people who want writing help in their daily workflow without switching tools. Number six is Sudowrite. This one's made
Number six is Sudowrite. This one's made for creative writers, such as novelists and storytellers. It helps with characters' development, plot twist, and rewriting the scenes. If you write fiction, this is a secret weapon.
Seventh is Ink Editor. It combines AI writing with SEO optimization. So, if you want your content to rank while sounding human, this tool does both.
Great for bloggers, content marketers. Coming to number eight is Hypotenuse AI. It is like a smart autocomplete that works across your browser. It helps you draft emails, social posts, and even essays. At number nine, we have Got
Writer. Clean, fast, and super affordable. Writer is a solid tool for small businesses and freelancers who want quick content without any complexity. And finally, number 10 is Wordtune. It
is not a full-blown writing platform, but it's amazing at rewriting, rephrasing, and polishing your text. Great for improving tone, clarity, and shortening long sentences. All right. Now, let's move on to into
the most powerful category, which is AI video creation. Whether you're a content creator, marketer, educator, or just someone who wants to make videos without cameras, editing headaches, these tools can seriously level up your workflow. In
the first place, we have got Pictory. This tools turns scripts or blog posts into professional-looking videos in minutes. It picks stock footage and add
minutes. It picks stock footage and add music and even guarantees subtitles all automatically. Next up at number two is Synthesia. It's famous for its AI avatars that look and sounds like real people. Just type in your script, pick
an avatar, and boom, you have got your studio-style video with zero filming. In the third place is Runway. This one's creative powerhouse. You can remove the video backgrounds, edit scenes, and text commands, and even generate a new video
content using Gen-2. It's specially great for short-form content or experimental visuals. Coming at number four is InVideo. It's built for marketers, content creators who want to make polished promo videos, reels, or
even YouTube content. Tons of templates, stock assets, and AI script assistance stock assets, and AI script assistance makes it fast and easy. Fifth is Lumen5. Ideal for turning blog posts or articles into animated videos. You just paste in
your text, and it automatically adds visuals, animations, and transition to match your message. Great for social media and explained content. At number
six, we have Descript. While it is more known for podcasting and editing audio. known for podcasting and editing audio. Its video editor is crazy useful. You can edit videos by editing the transcript. It's great for talking head
content, interviews, and repurposing long-form recordings. Number seven is veed.io, a browser-based editor that is surprisingly powerful. You get subtitles, transition, audio cleanup,
and more. All with a super clean UI. It's great for social media content and quick edits on the fly. Eighth is rephrase.ai. This one lets you create personalized video messages using AI avatars. Perfect
for sales, onboarding, or sending custom messages at scale without having to record anything manually. In the ninth place, we have Steve.ai. Super beginner-friendly, it turns simple text into videos using animations and
voice-over. Great for explain videos, internal training, or educational content. And finally, number 10 is elai.io. It creates AI avatar videos from just a prompt or script. Similar to Synthesia,
but with a focus on e-learning and corporate training content. Scalable, clean, and no need of production crew. Now, let's get into the most game-changing areas of AI, coding assistance. Whether you are a beginner
or a pro developer, these tools can help you write better code, debug faster, and even learn new languages or frameworks without getting overwhelmed. In the
first place, we have GitHub Copilot. Powered by OpenAI, this tool is a autocomplete on steroids. It suggests whole functions, helps you write whole functions, helps you write boilerplate, and even explains code, all
from inside your editor. If you're coding on a daily basis, this is a must have. Second is Codium. It's a free, fast, and supports a wide range of languages. It works like a co-pilot, but adds some extra customization and a lot
of devs love it for its speed and clean interface. At number three is Replit Ghostwriter. Especially great for beginners and educator, Replit's AI can
help you code right inside the browser. No setup needed. It also helps you explain errors and teach you as you build. Fourth is Tabnine. One of the easiest code compilation tool. Tabnine runs locally for privacy and works
across multiple IDEs. It's great for developers who want predictive coding without sending data to the cloud. In the fifth place, we have Amazon CodeWhisperer. Designed us to integrate smoothly with AWS tools and workflows.
It's perfect if you're building cloud-native application or working on AWS ecosystem. Coming at number six is Codex by OpenAI. It's a underlying model
behind co-pilot and if you're working in an experimental or custom environment, it offers a deep integration potential for AI-driven apps. Number seven is Mutable AI. A newer player focused on
speeding up your workflow with smart suggestions, automatic doc generation, and code refraction. It's great for clean, productive-level code. Eighth is Kite. It is one of the OG AI assistants for Python and still holds up if you
want lightweight, fast suggestions without overwhelming features. At number nine, we have Sourcegraph Cody. It's made for enterprises terms that work on massive code bases. It gives powerful contextual help and scales well across
contextual help and scales well across big projects. And finally, number 10 is AskCody. It's super handy for generating snippets, SQL queries, and even API docs. A nice tool for quick task when you don't want to dive too deep. And
that wraps up the top AI tools across image creation, voice generation, writing, video production, and coding. These tools aren't just cool. They actually help you get things done faster, better, and with less effort. If
you found this helpful, drop a like, subscribe, and let me know in the comments which tool you're excited to try or if there is anything that you love that I missed. &gt;&gt; So, now what is RNN?
RNN are a type of neural network that are designed to process sequential data. They can analyze data with temporal dimension such as time series, speech, and text. RNN can do this by using a hidden state
passed from one time step to the next. The next hidden state is updated at each of the time step based on the input and the previous hidden state. RNN are able to capture short-term dependencies in sequential data, but they struggle with
capturing long-term dependencies. Why the LSTMs are made. So, moving forward, let's discuss types of LSTM gates. So, LSTM models have three types of gates. The input gate,
gates. The input gate, the forget gate, and the output gate. So, let's first discuss the input gate. The input gate controls the flow of information into the memory cell deciding what to store.
The input gate determines which values from the input should be updated in the It uses a sigmoid activation function to scale the values between zero and 1 and then applies point wise multiplication to decide what information to store.
Next is forget gate. Controls the flow of information out of the memory cell deciding what to discard. should be discarded from the memory cell. It also uses a sigmoid activation
function to scale the values between 0 and 1 followed by point wise information to forget. The last one is output gate. Controls the flow of information out of the LSTM
deciding what to use for the output. The output gate determines the output of the LSTM unit. It uses a sigmoid activation function to scale the values from 0 to 1 then applies point wise multiplication to produce the output of
the LSTM unit. So these gates implemented using sigmoid function are trained using back propagation. They open and close based on the input and the previous hidden state allowing the LSTM to selectively retain or discard
information effectively capturing long-term dependencies. So now let discuss application of LSTM. LSTM models are highly effective and used in various application including video analysis, analyzing video frames
to identify action, object and scenes. The second is language simulation. Tasks like language modeling, machine translation and text summarization. The third one is time series prediction. So predicting future values in a time
The fourth is voice recognition. Tasks such as speech to text Tasks such as speech to text transcription and command recognition. The last one is sentiment analysis. Classifying text sentiment as positive,
negative or neutral. So there are many more examples of LSTM. So now let's move forward and understand LSTM model and how it works with example. Let's consider the task of predicting the next word in a sentence. This is a common
application of LSTM networks in natural language processing. So I will break it down a step by step using the analogy of remembering a story and deciding what comes next based on the context. So, imagine you are reading a story. As you
read, you need to remember what has happened so far to predict what might happen next. So, let's illustrate with a simple example sentence. The cat sat on the dash. So, you want to predict the next word, which could be mat or roof or
something else. An LSTM network helps us make prediction by remembering important parts of the story and forgetting irrelevant details. So, now let's dive into a step-by-step process. So, step-by-step explanation
using LSTM. The first one is reading the story, input sequence. As you read each word in the word sentence, you process it and store relevant information. For example, the you understand it's a determiner. Cat, you know it's a noun
and the subject of the sentence. Sat indicates the action performed by the subject. On, preposition indicating the relationship between the cat and the next noun. So, this sequence diagram showing the words being read and
processed. So, second comes forget gate. As you move through the sentence, you longer important. For instance, you might decide that knowing the is less important now that you have cat
and sat. So, the word forget gate helps discard this less important information. So, this sequence diagram you can see on the screen showing how relevant information is discarded.
So, the third one, input gate. When you read on, you need to decide how relevant this information is. So, this sequence diagram in the screen is showing how new information is integrated with the last one.
Okay, the fourth one is the cell state, memory part. So, this is like your memory of the story so far. It carries information about the subject cat and the action sat on. So, it updates the new information as you read each word.
Okay, the cat sat on the retaining the important context. So, this sequence diagram showing how the memory is updated with the new information. So, the last one is output gate. When you need to predict the next word,
the output gate helps you decide based on the current memory cell state. So, it on the current memory cell state. So, it uses the context the cat sat on the So, the predict the next word might be mad because cat and mad are often
associated with the context. So, it can predict anything. The cat sat on the table or on the sofa, anything. But, mad Why I'm saying mad? Because cat and mad are often associated in the same context.
So, this diagram is showing the prediction of the next word based on the current memory. So, there are many applications where you can use LSTM in the sentence. So, by the using of LSTM gates, input
gate, forget gate, and output gate, and updating the cell state, the network can predict the next word in a sequence by maintaining relevant context and discarding unnecessary information. This step-by-step process allow LSTM network
to effectively handle sequence and make accurate prediction based on the documentation and you could see the new features introduced with the chat GPT-4o. So, these are the improvements.
Uh one is the updated and interactive bar graphs or pie charts that you can create. And these are the features that you could see here. You could change the color, you could download it. And what we have is you could update the latest
and Microsoft OneDrive. And we have the interaction with tables and charts in an new expandable view that I showed you here that is here. You can expand it in the new window. And you
presentations and documents. Moreover, you can create a presentation also. That we'll see in further. And here we have how data analysis works in ChatGPT. You could directly upload the files from
Google Drive and Microsoft OneDrive. I will show you guys how we can do that and where this option is. And we can work on tables in real time. And there we have customized presentation, ready charts. That is you
charts based on a data provided by you. And moreover, a comprehensive security and privacy feature. So, with that guys, we'll move to ChatGPT. And here we have the ChatGPT 4.0
So, before commencing guys, there's a aspiring data analyst looking for online training and graduating from the best universities or a professional who elicits to switch careers with data
analytics by learning from the experts, then try giving a shot to Simply Learn's Purdue Post Graduate Program in Data Analytics in collaboration with IBM. You and pin comment. So, let's get started with data analysis
part. So, this is the pin section or the insert section connect to Google Drive, connect to Microsoft OneDrive, and you can upload it from the computer. This option was already there. That is
upload from computer. And you can upload at least or at max the 10 files. That Excel files or documents. So, the max limit is 10.
And if you have connected to Google Drive, I'll show you guys. Drive, I'll show you guys. Uh I'm not connecting you. could upload it from there also. And there's another cool update. That is
ability to code directly in your chat. Uh so, while chatting with ChatGPT, I'll show you guys how we can do that. And you could find some new changes that is in the layout. So, this is the profile
section. It used to be at the left bottom, but now it's moved to the top right and making it more accessible than ever. So, let's start with the data analysis part and the first thing we need is
data. So, you can find it on Kaggle or you could ask ChatGPT 4.0 to provide the data. I'll show you guys. So, this is the Kaggle website. You can sign in here and click on data sets. You can find all
computer science, education, classification, computer vision. Or else, you could move back to ChatGPT and you could ask the ChatGPT 4.0 model to generate a data and provide it in Excel format. So, we'll ask him
ask him "Provide a data set "Provide a data set that I can use for data analysis and provide in
CSV format." So, you could see that it has responded that I can provide a sample data set and he has started generating the data set he has started generating the data set here.
only 10 rows and he's saying that I will now generate this data set in CSV format. First, he has provided the visual presentation on the screen and now he's generating the CSV format. So, if you want more data, like if you want
100 rows or 1,000 rows, you could specify in the prompt and ChatGPT will generate that for you. So, we already have the data. I will import the data. You could import it from here. Or else, you can import it
from your Google Drive. So, we have a sales data here. We will open it. So, we have the sales data here. So, the first step we need to do is data cleaning. So, this is the crucial step
to ensure that the accuracy of our analysis is at its best. So, we can do that by handling missing values. That is missing values can distort our analysis. And here ChatGPT-4 can suggest methods to impute these values such as using the
mean, median, or a sophisticated approach based on data patterns. And handling the missing values, we will remove duplicates and outlier detection. remove duplicates and outlier detection. So, we'll ask ChatGPT
clean the data if needed. that would be clean the data if needed. And this is also a new feature. You can
see the visual presentation of the data here that we have 100 rows here and the columns provided that is sales ID, date, product category, quantity, and price per unit, and total sales.
So, this is also a new feature that Okay. Uh we just head it back. We'll move back to our ChatGPT chat We'll move back to our ChatGPT chat here.
So, you would see that ChatGPT has cleaned the data and he's provided that it has checked for missing values, checked for duplicates, and ensured checked for duplicates, and ensured consistent formatting. And he's saying
Okay. Okay. So, now we'll ask him that Okay. So, now we'll ask him that execute
and provide the clean data. As ChatGPT has provided that these would be the steps to clean the data. And let's see. So, he's provided a new CSV file
with the clean sales data. We will download it download it and ask him to use the same file only. new cleaned sales data CSV file
what analysis we can do further. But, once our data is clean, the next step is visualization. So, visualizations help us understand the representation. So, the first thing we will do is
we will create a prompt for a generating the histograms. And we'll do that for the age distribution part. So, we'll write a prompt that generate a So, we'll write a prompt that generate a histogram.
Generate a histogram to visualize the distribution of customer ages. Visualize the distribution
code button. If you just select the text and you would find this reply section, just click on that and you could see that it has selected the text or what you want to
to get or the prompts started with ChatGPT. make it cross and you could see that it has provided the histogram here. And these are the new features here.
we could see that he is providing a notification that interactive charts of this type are not yet supported. That is option. I will show you the color change option
bar chart section. So, these features are also new. You can download the chart from here only. And this is the expand chart. If you click on that, you could see that you could expand the chart here and continue chat
with ChatGPT here. So, this is the interactive section. the histogram that is showing the age range are from 18 to 70 years with the
distribution visualized in 15 bins that he has created 15 bins here. And now moving to another visualization that we'll do by sales by region. So, before that I will open the CSV file
that is provided by the ChatGPT. So, you guys can also see what data he So, you guys can also see what data he has provided.
you could see that we have columns sales ID, date, product category, quantity, price per item, total sales, region, and sales person. So, now moving back to ChatGPT. So, now we'll create a bar chart showing total
sales by region. So, we'll enter this prompt that create So, we'll enter this prompt that create a bar chart
creating bar charts or histogram charts, but we can do that for only two columns. If you want to create these data visualization charts, we need two So, you could see that he has provided the response and created the bar chart
And this is the interactive section. You could see that there's an option to that, we can't like we are not getting any
information we scroll on that. And if I enable this option, you could see that I can visually see how many numbers this bar is indicating. And after that we have the change color section.
provided. So we can change it to any color that is provided here or you could just write the color code here. that is download and another is the expand chart section.
And if you need uh what code it has done to figure out this bar graph, so this is the code. You could use any ID to do so. You could use any ID to do so. If you don't want the presentations or
the visualizations of the bar charts here, you could use your ID and use the Python language and it will provide the code for you. Just take your data set and read it through pandas and generate the bar
through pandas and generate the bar charts. category-wise sales section. So here we'll generate a pie chart showing the proportion of sales for each product category.
category. So for that we'll write a prompt.
generating the pie chart. And this is also an interactive section. If you click on that, you would be seeing a static pie chart. And if you want to change the color, you can change for any section that
would be clothing, electronics, furniture, or kitchen. And similarly, we have the download section and the expand chart section. So, this is how this new ChatGPT 4.0 model is better than ChatGPT 4.
That you could use a more interactive pie charts. You could change the colors for that. And you can just hold over these bar charts and them. So, after this data visualization, now
we'll move to statistical analysis. So, this will help us uncover patterns and So, the first thing we'll do is correlation analysis. And for that, we'll write the prompt, "Analyze the correlation between age and purchase
amount." So, this correlation analysis helps us understand the relationship between two variables. So, this can indicate if older customers tend to spend more or less. So, we'll find out that
by analyzing the data, and we'll provide a prompt to ChatGPT that analyze the correlation between age and purchase amount.
You could see a scatter plot that shows the relationship between customer age and total sales. That is with a calculated correlation coefficient of approximately 0.16. So, this indicates a weak positive correlation between age
and purchase amount, suggesting that as customer age increases, there's a slight well. So, you could just see the scatter plot That if the age increases, so it is not correlated
to sales, as you could see an empty graph here. graph here. So, till 40 to 50 years of age or the 70 years of age, you could find what amount they have spent here. That
is the total sales accumulated by these ages. So, now moving to sales trend. So, here we'll perform a time series given dates. So, what does this do is
time series analysis allows us to examine how sales amount changes over time, helping us identify trends and seasonal patterns. seasonal patterns. So, for that we'll write a prompt.
time series analysis of purchase amount
provided us the response, and here's the time series plot showing total sales And each point on the plot represents the total sales for a particular day. So, through this you can find out and
the businesses find out which is the seasonal part of the year and where to stock up their stocks for these kind of dates. And after that, you could also do customer segmentation. So,
what does this do is So, we can use clustering here to segment customers amount. So, clustering groups customers into segments based on similarities. This is useful for targeted marketing and
personalized services. And after that, we have the advanced usage for data analysis. Here we can draw predictive modeling table and do the market basket analysis and perform a customer lifetime value analysis.
So, we will see one of those and what we'll do is we'll perform a market basket analysis and perform an association rule mining to find frequently bought together products. So,
the theory behind this is the association rule mining helps identify purchased together aiding in inventory management and cross-selling strategies. So, for that we'll write a prompt that
find frequently bought together products. So, for that we'll write a prompt here. So, for that we'll write a prompt here. Perform an association rule mining
to find frequently bought for this prompt what does ChatGPT for a
this prompt what does ChatGPT for a response to us? But, we don't need a code here. We need the analysis.
provide code. Do the market basket analysis
and provide visualizations.
provided the response that given the limitations in this environment, so he's not able to do the market basket analysis here. So, but he can help us how we can perform this in an ID.
required libraries, then prepare the data, and here he's providing the So, you could see there are some limitations to ChatGPT 4 also, that he can't do advanced data analysis.
So, you could use the code in your ID and do the market basket analysis there. So, there are some limitations to ChatGPT 4 also. And now we'll ask ChatGPT, "Can you create a presentation based on
the data set?" And we'll provide a data set to it also. So, we'll provide a sample sales data.
"Can you create a presentation or PowerPoint presentation
and only provide data visualization
started analyzing the data. And he's stating that and he will start by creating a data visualization from the provided data set and compile them the provided data set and compile them into a PowerPoint presentation.
provided us the response. And these are all the presentations or the bar graphs that he has created. And now we have downloaded the presentation here. We'll open that.
that is created by chat GPT-4. &gt;&gt; Before you bring generative AI into the picture, let's think about how data analytics is usually done. Traditionally, if you wanted to know how sales changed from one quarter to
another, you would need to open your database, then write a SQL query or some pandas code, then generate a chart in Excel or Power BI, and then finally interpret the result by yourself.
requires strong technical skill. Now, imagine you don't have to do all that. Instead, you just ask a simple question in English. What was the sales growth in quarter two compared to quarter one? A generative AI system will write the SQL
or Python code for you. And then it is going to run it against the data. And also, it is going to show you the percentage change and even it can give you a short explanation. This means that
somewhat analytics have become conversational, automated, and predictive. You talk to your data the way you talk to a colleague. Now, you'd be wondering, why does this matter for data analyst? Because it saves time,
reduces technical barriers, and lets you focus on understanding and acting on mechanics of coding. GenAI is not here to replace analyst, but to act as a co-pilot speeding the
queries, cleaning data, building visualizations, and even suggesting Now, there are some of the core applications of GenAI in data analytics. insights. So guys, as you can see all over here,
that traditionally, if you wanted to get an answer from a data, you had to write your own SQL queries or Python code. That means only trained data analyst could do it. But with GenAI, anyone, even a marketing
manager or HR professional, can simply type a question in plain English. For example, show me a chart of customer churn rate by region. And the AI is going to understand the request and write the code in the background. Run it
chart. Now, you can imagine how making analytics from the given data has become so much easier. Even when you don't know coding. Next one, if you could see, you are
going to get automated data storytelling. Numbers on the charts are very useful. But they don't always tell the full story. GenAI goes a step further. It can explain numbers in words. For example,
instead of just showing a revenue chart, GenAI can automatically write revenue increased 12% in APAC, mainly due to surge in mobile app users. So, you can see how it is saving time for analyst and also it is helping
non-technical people. Third one is predictive and prescriptive analytics. Most traditional analytics tools tell us what has already happened. But GenAI can
look at patterns in the data and help predict what might happen next. For example, you can ask something like this, like based on the current trends, underperform in the next quarter? GenAI can not only make the prediction,
GenAI can not only make the prediction, but also suggest what actions to take to increase the promotion in a particular region. Next is data cleaning and preparation. Every analyst knows the hardest part of analytics is not the
charts. It's cleaning messy data. values or spelling mistakes or duplicates, then it becomes pretty harder. But GenAI can automatically suggest
cleaning steps, even generate the code to fix the data set. For example, guys, if customer data has duplicate entries, GenAI can detect and remove them if revenue numbers are missing, and it can
fill them based on the relative values. This saves hours of manual effort. And finally, you can get visualizations on demand. In traditional BI tools, you
drag and drop fields, adjust filters, and then format charts. This can take a lot of clicks, but with GenAI, you can just ask, "Create a heat map of sales by product category and region." And boom, you're going to get
it very much easily. The AI is instantly going to generate a professional-looking chart or even a full dashboard. This makes data visualization faster, easier, and more flexible. So guys, I hope so you would
have got a brief idea regarding the application of GenAI in data analytics. Now, let us begin to the next part, that is the hands-on tutorial. So guys, in
most of the real case scenario, you are going to have a pre-prepared data. It may be structured, semi-structured, or unstructured. But for this tutorial, we are going to create a random data for the product
based on the given region and also the date and time. Also, if you want to try the simulation to generate a random data based on the given dates, regions, and the products. So, you're going to get a code something
like this. Now, let us dive deep into this given tutorial and try to understand what are the libraries used in this given code. You can see it has used Pandas, NumPy, and Matplotlib. And also, guys, you can
see it is trying to create the data with the help of NumPy. So, you can see there are dates, regions, and products. Then there is a for loop, and it is rows. So, you can see with date, region,
products, unit sold, price, and revenue. And finally, you are storing this in a data frame. Now, let us try to run this code.
Now, let us move to the next step and try to do some analytics. Now, guys, let us give a prompt to chat GPT. Something like this, saying that I have pandas
data frame named sales with column date and region, and also with products, unit sold, price, and revenue. Try to give a brief uh idea about what data you have to chat GPT. And then say write simple pandas code to answer what is the
average revenue per region, or return a small table with column region and lowest. Let us try to hit the button, and let's see. Now, you can see all over here, we have got a code. So, we can just copy
this and try to paste on our collab. So, now let us try to run this, and you can see we have got the given output with the region and the average revenue, like west, south, north, east, and all those
things. Let us move to the next analytics part. But before that, I would like to ask you some questions. Now, here's a little part I want you to try. You can also ask all over here that show
the total revenue by product, and which day had the highest revenue overall. you. So, it it's mostly it's becoming a tool-based, and uh but with the complexity of the data,
things can become little bit complex. But since we have the GenAI, now your work will become very much easier. So, do try this, guys, and mention in the comment section below how many of you have got the right result. Now, as
we all know that real data is actually very messy. So, guys, I have added some impurities in the given data. You can see I have made some missing values, I have made some duplicate rows, and I have also some lowercase and spaces.
So, that makes a pretty interesting how GenAI is going to deal with this Let us try to hit the run button and first try to insert all those thing, and you can see the data has been added. Now, let us try to do some analysis.
Now, let's say in a given scenarios, first you need to understand your data. Where are the missing values and what are the issues in your given data? And then you can give prompt something like this. Say, I have pandas data frame
messy like before, please write a simple readable code and trim spaces and fix regions like title case, drop the duplicate rows, and fill the missing revenue with unit sold. And make sure your date is in proper
data type, and then return the clean data frame named clean. Now, let us give another prompt to generate a chart with the help of GenAI. Basically, I'm using and the model is five. So, you can write a prompt something
like this. Write a simple matplotlib code to plot a bar chart of the top five products by total revenue from the data frame clean, and also add the title and label the axis. So, clean is our data frame.
Let's click enter. You can see GPT has easily given us the code. Just copy go to the terminal and just paste the given code, given code, and just click on enter.
have got a very amazing visualization. You can see there is total revenue, revenue, and you can see this is watch, laptop, phone, headphones. So, in this simulation, you can see how easily you can generate charts.
very much interesting. So, let us say I asked the GPT something We'll keep the forecasting simple. We will use total revenue by month, then use a 3-month average to guess the next
one. So, you can type a prompt something like this, that we want forecasting, we will use total revenue by month, then use a 3-month average to guess the next one. Just click enter after that. And you can see it is going to give you
the code, which is going to generate the forecasting. step, calculate the total revenue by month, then calculating the 3-month moving average, and finally, it has done the forecasting of the next month.
Now, let us copy this given code and paste on the Google Colab. So, we have copied our given code. Let's try to run it.
have got the output, and it has generated the forecasting of our given product. So, you can see from this graph, guys, this is the revenue Y axis, and on the X axis, we have the actual revenue, and the dash lines are 3-month
average. Now, you can see from all over here, we have the given years, like 2024, first month, the third month, fifth month, seventh month, ninth month, And it is creating the monthly revenue
versus 3-month So, that is amazing analysis, and we have just got to do it in few seconds by writing the prompt. Now, you would be also thinking about the same thing which I am thinking, like
GenAI is definitely going to make a lot of things easier in the domain of data analytics. Hello, everyone, I am Mayank and welcome to today's video where we will be talking about LLM benchmarks, tools used to test and measure how well
large language models like GPT and Google Gemini performs. If you have ever wondered how AI models are evaluated, this video will explain it in simple terms. LLM benchmarks are used to check how good these models are at tasks like
coding, answering questions, and translating languages, or summarizing text. These tests use sample data and a specific measurement to see how well the model perform. For example, the model might be tested with a few examples,
like few-shot learning, or none at all, like zero-shot learning, to see how it handles new tasks. So now the question arises, why are these benchmarks important? They help developers understand where a model is strong and
where it needs improvement. They also make it easier to compare different models, helping people choose the best one for their needs. However, LLM benchmarks do have some limits. They don't always predict how well a model
will work in real-world situation, and sometimes models can overfit, meaning they perform well on test data, but they struggle in practical use. We will also different models based on their benchmark scores, giving us a clear
picture of which models are performing the best. So stay tuned as we dive into how LLM benchmarks work and why they are so important for advancing AI. So without any further ado, let's get started. So what are LLM benchmarks? LLM
benchmarks are standardized tools used to evaluate the performance of large structured way to test LLMs on a specific task or question, using sample data and predefined metrics to measure their capabilities. These benchmarks
assess various skills such as coding, common sense reasoning, and NLP tasks answering, and text summarization. The importance of LLM benchmarks lies in their role in advancing model development. They track the progress of
an LLM, offering quantitative insights into where the model performs well and feedback is crucial for guiding the fine-tuning process, allowing researchers and developers to enhance model performance. Additionally,
benchmarks offers an objective comparison between different LLMs, helping developers and organizations choose the best model for their needs. So, how LLM benchmarks work? LLM benchmarks follow a clear and systematic
process. They present a task for LLM to complete, evaluate its performance using specific metrics, and assign a score based on how well the model performs. So, here is a breakdown of how this process work. The first one is setup.
LLM benchmarks come with pre-prepared sample data, including coding challenges, long documents, math problem, and real-world conversation. The task expand various areas like common sense reasoning, problem-solving,
question-answering, summary generation, and translation, all presented to the model at the start of testing. The second is step is testing. The model is tested on one of the three ways. Few-shot, the LLM is provided with a few
examples before being prompted to complete a task, demonstrating its ability to learn from limited data. The second one is zero-shot. The model is asked to perform a task without any prior examples, testing its ability to
understand new concept and adapt to unfamiliar scenarios. The third one is fine-tuned. The model is trained on a data set similar to the one used in the benchmark, aiming to enhance its performance on the specific task and
work. The third step is the scoring. So, after completing the task, the benchmark expected answer and generates a score, typically ranging from 0 to 100, reflecting how accurately the LLM performed. So, now let's moving forward,
let's see key metrics for benchmarking LLMs. So, LLM benchmark uses various metrics to assess performance of large language model. So, here are some commonly used metrics. The first one is accuracy or precision. Measure the
the model. The second one is recall, also known as sensitivity, measure the number of true positive reflecting the correct prediction made by the model. The third one is F1 score, combines both accuracy
and recall into a single metric, weighing them equally to address any false positive or negative. F1 score ranging from zero to one, where one indicates perfect precision and recall. The fourth one is exact match. Tracks
the percentage of predictions that exactly match the correct answer, which is especially used for the task like translation and question answering. The fifth one is perplexity gauges. Here, it will tell you how well a model
predicts the next word or token. A lower perplexity score indicates better task comprehension by the model. The sixth one is BLEU, Bilingual Evaluation machine translation by comparing n-grams, sequence of adjacent text
element, between the model's output and the human produced translation. So, these quantitative metrics are often combined for more thorough evaluation. So, in addition, human evaluation introduces qualitatively factors like
coherence, relevance, and semantic meaning to provide a nuanced assessment. However, human evaluation can be time-consuming and subjective, making a balance between quantitative and qualitative measures important for
comprehensive evaluation. So, now let's moving forward see some limitation of are valuable for assessing model performance, they have several fully predicting real-world effectiveness. So, here are some few.
The first one is bounded scoring. Once a model achieves the highest possible scores on the benchmark, the benchmark loses its utility and must be updated with more challenging tasks to remain a meaningful assessment tool. The second
one is broad data set. LLM benchmark often rely on sample data from diverse subject and task. So, this wide scope may not effectively evaluate a model performance in such cases, specialized fields, or specific use cases where more
tailored data would be needed. The third one is fine-grained assessment. Benchmark only test a model's current skills, and as LLMs evolve and the new capabilities emerge, new benchmarks must be created to measure these
advancements. The fourth one is overfitting. So, if an LLM is trained on can be lead to overfitting, where the model performs well on the test data but it struggles with the real task. So, this result is scores that don't
truly represent the model's broader capabilities. So, now what are LLM leaderboards? So, LLM leaderboards publish a ranking of LLMs based on a provide a way to keep track to the myriad LLMs and to compare their
performance. LLM leaderboards are especially beneficial in making decisions on which models to use. So, here are some. So, in this you can see here are some. So, in this you can see here OpenAI is leading and GPT-4o second
and the Llama third with 405 parameter B and 3.5 Sonnet is there. So, this is best in multitask reasoning. What about the best in reasoning. What about the best in coding? So, here OpenAI 01 is leading. I
guess this is the Orion 1, and the second one is 3.5 Sonnet, and after that in the third position there is GPT-4o. So, this is in best in coding. The next comes fastest and most affordable models.
So, fastest models are Llama 8B parameter, 8B parameter. And the second parameter, 8B parameter. And the second one is LLaMA Llama 70B. And the third one is 1.5 Flash. This is Gemini 1. And lowest latency.
And here, it is leading Llama again. In cheapest models, again Llama 8B is leading. And in the second number, we have Gemini Flash 1.5. And in third, we have GPT-4o mini.
Moving forward, let's see standard benchmarks between Claude 3 Opus and GPT-4. So, in general, they are equal. In reasoning, Claude 3 Opus is leading, and in coding, GPT-4 is leading. In math, again, GPT-4 is leading. In tool
use, Claude 3 Opus is leading, and in multilingual, Claude 3 Opus is leading. &gt;&gt; So, let's talk about the very first top AI tool, which is Skillora.ai. This AI tool aims to provide the best AI-driven
mock interview experience. You can take mock interviews for any job roles and subject, receiving instant constructive feedback. Skillora's AI interviewer listens to your responses and asks dynamic flow of questions to test your
depth of knowledge. With the essential plan, you gain access to six AI mock interviews per month. You'll receive instant feedback, ideal answers, and performance reports, helping you upgrade your skills for just
$8 per month. The premium plan allows you to enjoy 15 AI mock interviews per month. You'll get the same instant feedback, ideal answers, and performance reports, along with priority support, maximizing your learning for only $15
per month. Next up, we have Viscose. Viscose's advanced AI coach is designed to elevate your job interview skills, harnessing insights from over 10,000 It offers tailored sessions that simulate real interviews, providing
personalized feedback and performance analytics. It transcribes your performance and delivers a conversational experience akin to an actual interview. It also offers customized feedback and performance
analytics, helping you refine both your answers and communication skills. During practice session, you can craft questions specific to your desired role and industry, ensuring relevance and depth in your preparation, supporting
various interview styles. The price for Eva's services are available for free. Next up, we have interviewing.io. Interviewing.io is an AI-powered platform that provides candidates with
simulated technical interviews. The platform uses machine learning algorithms to access your coding skills, problem-solving abilities, and communication style. You can get four simulated technical interviews for $225.
Next up, we have Mockmate. Mockmate is an AI powered tool that allows you to practice answering common interview questions. This tool provides feedback on your answers and gives tips on how to improve your performance. The service is
available for $29 per month. Moving on to ChatGPT. ChatGPT is an AI chatbot that can be used to practice answering interview questions in a conversational setting. The chatbot can be customized to
interview, and it provides feedback on your answers. The free plan allows you to get started with writing assistant, problem-solving, and many more. You will have limited access to ChatGPT 4o mini, including data analysis, file uploads,
vision, web browsing, and custom GPTs. All of this is available at no cost. Then we have the plus plan, which is for $20 per month. The plus plan amplifies your productivity by offering early access to new features, access to
access to new features, access to ChatGPT 4 4o and GPT 4o's mini, up to 5x more messages for ChatGPT 4o. You can access to data analysis, upload files, vision, web browsing, and DALL-E image generation. You can also create and use
custom GPTs. Then we have My Interview Practice. My Interview Practice is an AI tool that allows you to practice interviews in a simulated environment. The platform uses AI to generate interview questions based
on your chosen field, and provides instant feedback on your responses. This tool is particularly useful for those who want to practice their interview skills in low-pressure environment. The plus plan is for $49 per month. You get
1 month of access. And the premium plan is for $57 per month and you will get more benefits. Next up, we have Yoodli. Yoodli offers
real-time interview coaching using AI. It analyzes your speech and body language, providing feedback on how to improve your performance. Yoodli's services are available for free. Now, you might be wondering how AI tools
can help you ace your job interviews. AI tools can assist in several ways. At first, we have research companies. AI tool provides information about companies' website, news articles, and social media, helping you tailor your
answers to the interview's question. Practice questions. AI tool help you practice answering common interview questions, boosting your confidence and preparation. Feedback. AI tool offers feedback on
improvement, and refining your responses. Personalized advice. AI tools provide tips and advice on improving your interview skills, giving you an edge to your next job interview. Choose the
right AI tool and select an AI tool that fits your needs and preferences. Use the tool consistently, starting a few weeks before your interview to maximize its benefits. After practicing with the AI tool, get feedback from others to
identify areas for further improvement. Dress professionally and be punctual for your interview to make a positive first impression. Show enthusiasm and confidence during the interview to leave a lasting impression on the interviewer.
AI tools can significantly aid in interviewing preparation. By using them effectively, you can enhance your chances of success in your next job interview. Hey everyone, have you noticed how the word AI is being thrown
around almost everywhere today? From marketing posts to casual conversations, people are saying things like, "We use AI. We built an AI tool." or "AI is the future." But, here's the truth. Most of these people have no real understanding
of what AI actually is and that's the problem because AI is not a magic. It's not just a trend. It's built on real foundations, mathematics, logic, and code. If we are serious about working in AI or building with AI, we need to
understand the core of how it actually works and that's exactly why we are making this video. The goal here is very simple, to bring clarity. So, in this video, we will give you 15 basic questions about AI. Not advanced stuff,
not industry buzzwords, just the real absolute fundamentals that every beginner or aspiring AI professionals must know. No fluff, no jargons, and no shortcuts. These questions are meant to check if you truly understand what AI
actually is, how it works, and what's going on under the hood. So, if you don't know the answer to some of these, that's okay. It just means you now know exactly what to go back and learn because the truth is if someone can't
answer these 15 basic questions, they are not ready to work in AI yet and that's not gatekeeping. That's just being honest about what this field requires. Now, before we dive into the core questions, take a moment to really
think about each one. Don't rush through them. Make sure you subscribe to our channel and drop a like and share if you want us to make more such videos. Let's get into it. Let's have a look at our first question, which is what does it
actually mean for a machine to learn? Now, here's a quick explanation of this question. We say machines learn from data, but what does learning really mean data, but what does learning really mean in this context?
understand anything or it is just advanced pattern recognition? Let's have an explanation to this question. Now, this question challenges whether AI has any kind of awareness or if it's just calculating probabilities.
would happen if we trained an AI model on completely incorrect or biased data? Now, this question isn't just theoretical. This happens in the real theoretical. This happens in the real world. Think about the consequences.
build an AI that makes decisions without being explicitly told how? Now, I'll give an explanation of this question. So, this goes to the heart of what intelligence mean in machines and whether rules are always required.
do you know if an AI system is being smart or just memorizing? Let's have an explanation of this question. Now, this is a real challenge in evaluating AI performance. Is the system learning concepts or just recalling patterns? You
concepts or just recalling patterns? You need to know about this. question, which is, what is overfitting and why is it a problem? Now, you can
think of it as a model that memorizes instead of learning can feel badly in instead of learning can feel badly in real world.
is, what does an activation function do in a neural network? Let's have an explanation to this question. It's a small component with a huge impact on small component with a huge impact on how neural networks work.
learn from completely random data? Now, this question can be tricky because it makes you think about how learning really works.
descent and why do we use it? Now, you can think of this question about if you have ever trained a model, you have probably used this, but do you probably used this, but do you understand it or you not?
and the quality of the data set matter so much? Now, the explanation for this question is, you can't make something smart with garbage input, and yet many smart with garbage input, and yet many try.
difference between a model and an algorithm? Now, you can think of it like this. These two get confused a lot, but they are not the same thing.
model performs well on training data but fails on real-world data? Now, for this question, this scenario is more common than you would think, and it would be than you would think, and it would be way more dangerous.
function? Let's have an explanation to this question. It's not just some technical detail, it literally tells the model how wrong it is.
Question number 14. Can a machine learning model be 100% accurate? Now, this is one of those questions that seems simple, but the implications are seems simple, but the implications are deep.
in AI and where do they come from? Let's have an explanation to this question. Now, this question goes beyond math and into real-world consequences.
of our course. If you have any doubts or questions, ask them in the comment section below and our team of experts will be happy to help you as soon as possible. If you found this course helpful, do like this video and don't
forget to subscribe for more such courses. Until next time, thank you for watching and keep learning with Simply Learn.
