Why Python is the best for automation
55sHigh educational value with a clear, relatable problem-solving hook for beginners.
▶ Play ClipAI Summary
This video demonstrates how to automate file management using Python, specifically by creating a script that organizes photos from a Dropbox camera upload folder into a folder structure sorted by year and month. The creator walks through the entire process, from understanding the requirements to implementing the script and scheduling it to run automatically.
Chapters
[00:00]
Introduction to Python Automation
The video introduces the concept of using Python scripts to automate tasks, emphasizing the importance of making computers work for you. The creator mentions that Python is chosen for its cross-platform compatibility and strong file system support.
[00:43]
The Photo Organization Script
The main example is a script that moves photos from a Dropbox camera upload folder into a new photos folder sorted by year and month. The script runs automatically in the background, eliminating manual organization.
[01:08]
Thinking Process and Requirements
The creator explains the thought process: check files, determine if they are photos, extract date information from file names or creation dates, create target folders, and move files while handling duplicates.
[03:20]
Script Overview and Folder Structure
The script iterates through files, transfers them to appropriate folders, and uses a naming strategy with month numbers to ensure alphabetical sorting.
[04:08]
Setting Up Variables and Listing Files
The code defines source and target folder variables, uses os.listdir() to get files, and imports the os module.
[05:21]
Filtering for Photos and Videos
The script creates a list of allowed file extensions (JPEG, PNG, etc.) and checks each file's extension to filter only photos and videos.
[06:34]
Extracting Date from File Name
A function get_date() is defined to extract year and month from file names using regular expressions, handling patterns like YYYY-MM-DD.
[09:50]
Handling Files Without Date in Name
If the file name lacks a date, the script falls back to using the file's creation date, with platform-specific handling (Windows vs Mac/Linux).
[13:31]
Returning Year and Month
The get_date() function returns a dictionary with year and month, and includes a check for files without any date, returning zeros.
[14:27]
Creating Target Folder
The script constructs a folder name like '2022-08 August' and creates it if it doesn't exist, using os.makedirs().
[17:10]
Moving Files and Handling Duplicates
The script checks if the target file exists; if not, it moves the file using shutil.move(). If it exists, it compares file sizes to determine if it's the same file, and if so, removes the source; otherwise, it renames the file.
[19:36]
Scheduling Automatic Runs
The script can be scheduled using Task Scheduler on Windows or Cron on Mac/Linux. The creator shows a cron job that runs at minute zero every hour.
[20:31]
Recommended Resource
The creator recommends the book 'How to Automate the Boring Stuff with Python' by Al Sweigart for further learning.
The video provides a practical, step-by-step guide to building a Python script for file organization, demonstrating how to handle dates, create folders, and move files. It emphasizes the value of automation and offers a solid foundation for viewers to create their own scripts.
Mentioned in this Video
Tutorial Checklist
Study Flashcards (9)
What are the three reasons the creator gives for choosing Python?
easy
Click to reveal answer
What are the three reasons the creator gives for choosing Python?
easy
Click to reveal answer
Cross-platform compatibility, great file system support, and ease of use.
00:29
What is the purpose of the script demonstrated in the video?
easy
Click to reveal answer
What is the purpose of the script demonstrated in the video?
easy
Click to reveal answer
To move photos from a Dropbox camera upload folder into a folder structure sorted by year and month.
00:43
How does the script determine the year and month for a file?
medium
Click to reveal answer
How does the script determine the year and month for a file?
medium
Click to reveal answer
It first tries to extract the date from the file name using regex; if that fails, it uses the file's creation date.
01:36
What Python module is used to list files in a directory?
easy
Click to reveal answer
What Python module is used to list files in a directory?
easy
Click to reveal answer
os.listdir()
04:21
What is the regex pattern used to match dates in file names?
hard
Click to reveal answer
What is the regex pattern used to match dates in file names?
hard
Click to reveal answer
The pattern matches YYYY-MM-DD with optional hyphens, using \d{4} for year, \d{2} for month and day.
07:21
Why is the creation date handling platform-specific?
medium
Click to reveal answer
Why is the creation date handling platform-specific?
medium
Click to reveal answer
On Windows, os.path.getctime() returns creation time, but on Mac/Linux it returns modification time; so different methods are used.
10:18
What does the get_date() function return?
easy
Click to reveal answer
What does the get_date() function return?
easy
Click to reveal answer
A dictionary with keys 'year' and 'month'.
12:49
How does the script handle duplicate files?
medium
Click to reveal answer
How does the script handle duplicate files?
medium
Click to reveal answer
It checks if the target file exists; if so, it compares file sizes. If sizes match, it removes the source; if not, it renames the file.
18:23
What scheduling tools are mentioned for automating the script?
easy
Click to reveal answer
What scheduling tools are mentioned for automating the script?
easy
Click to reveal answer
Task Scheduler on Windows and Cron on Mac/Linux.
19:36
💡 Key Takeaways
💡
Real-world automation example
Shows a practical use case that viewers can relate to and implement.
00:43
🔧
Regex for date extraction
Provides a reusable regex pattern for extracting dates from filenames.
07:21
📊
Cross-platform file creation date
Highlights the platform differences in getting file creation dates, a common pitfall.
10:18
🔧
Duplicate file handling
Demonstrates a robust approach to avoid overwriting files.
18:23
⭐
Recommended book
Points to a valuable resource for further learning.
20:31Full Transcript
[00:00] However, to do that, you need to know how to make computers work for you. In this video, I'm covering one of the Python scripts every single month. On this channel, I'm teaching the skills to help people grow
[00:16] If you already know another language that is not Python, You don't have to use Python, but there's a few reasons why I choose to use Python.
[00:29] multiple operating systems without having to do anything. And 3. has great support for the file system, So the Python scripts I'm going to go through
[00:43] is the one that I use to take the photos from my Dropbox camera upload folder and moves them into a new photos folder sorted by year and month. to make sure my photos were organized.
[00:55] that runs automatically in the background, so I don't need to do anything. Now I want to walk through my thinking process when building this script, So let's get stuck in.
[01:08] and then with the files we have, we need to check That's all we really care about that we want to put in our photos folder. But if they are, then we need to see
[01:24] what can we use to be able to sort them by year and month. Now Dropbox tends to rename the files that it's putting into that folder with the date and time that they were added.
[01:36] if it has a date in it, to then extract what the year and month is. Now, if it doesn't have a date on it, we need to see when that file was created
[01:48] Now, in some cases, you can use things like the EXIF data, However, this requires a few extra packages in Python,
[02:00] to try and set up on Mac, so I'm glossing over it for now. by the data that's created, we're going to have a year and month. From there,
[02:13] Now we need to see, does that exist? If there isn't, then we need to create that folder. Next we need to see is the file that we want to copy across.
[02:30] If there's not great, we can just move the file to the new folder. then we need to do a few checks to see whether it's actually the same file. So for me, I'm just going to look at the size of the file.
[02:47] and it's exactly the same size, and chances are it's the same photo. if it's the same file, it's the same name, it's the same size. If it's not the same size file, then it's chances are it's a different photo
[03:05] So that's our script. That's what it's going to do. So this is what a Python script is going to do. from different points in the year.
[03:20] and it's going to transfer them across and put them into the right folder. So if we run this script So we now have in 2022, we now got different months
[03:37] and then the photos are in each of the different folders. Now I've got this sort of naming strategy where I've got the number of the months So when you sort the photos by a static order that appear in the right order.
[03:52] We need to know which folders we're going to be looking at. So to do this, I'm just going to create a couple of variables in our code. always can be using the same photos, it doesn't really matter.
[04:08] being our camera upload folder and the target being our photos folder. So to do that, we're going to create a new for new variable called files
[04:21] and we're going to use OS to list directory. And then look at this source directory. So what we need to do is important at the top of our code
[04:35] to be able to use it and just type import OS. To do that, we're going to use a fully 3 to 4 file in files.
[04:49] And then that's just what we have here to see what the files coming out for that. just to see what's happening and make sure we save it on our type But you might have just Python on yours, so let's see what that does.
[05:07] So we've got those images we've got we've also got a few other things They still appear in there. So next we need to work out whether this is going to be a photo or not.
[05:21] Some of them are JPEG, some are PNG. And we also want the videos as well as they might be a movie or MP for So to do that first we need to create a list
[05:35] So to do this, we're going to create an array So we're going to just create this at the top of the copy and paste. So here we have our JPEG, PNG JPEG, a different name
[05:50] So to check the extension rates, these files we can do if we do file level, so that we can be sure that we're going to get a match on that file extension.
[06:04] and here we can then either basically just put in a string, but we're going to use a tuple. Tuple and then how you pronounce it, R Right that we had earlier.
[06:19] Curl So next line we can then do print again with file and we should open well and get the pictures this time. So the next thing we need to do is to get the date
[06:34] or we can use the created data sitting on the file. for this in this call, define get date.
[06:46] that we're passing in as well as the file that we're looking at. So we have something in that. and we're going to put in source and file.
[07:04] We want the year and months, so let's do that. Reg, it's going to be a bit scary at first, but it's quite simple. just so you can see, and I'll explain to you what each of it does.
[07:21] to be able to detect that it's actually got this format of data in there. So the dot star basically means it matches any character.
[07:33] but it's actually somewhere else in the phone. I haven't got any pictures from the 1980s or 1990s in that. And then the slash D on here just means any number between
[07:49] zero and nine that's going to match any year after the year 2000. Question mark means zero or one. If neither of these hyphens are that, it will still match.
[08:04] which is obviously going to start either with zero or one And then we've got a slash date which matches everything from 0 to 9. Next, we've got another optional hyphen followed by the day.
[08:17] It's going to start either with zero, one, two or three, and then it will stick And then finally we've got a dot star at the end So it doesn't matter if it's got the time that we just going ignore it.
[08:31] so we've got a bracket here and another one here and another one here. So this matches the year, the month and the day. a pattern match on that file name that we can then pull out of the regex.
[08:50] and then we're going to use regex to match what we have in our fallen name with the one that we've got in our date pattern.
[09:02] I never go out and file. So we're just going to have to import already at the top. So next we need to pull out the year and the month from the dates if it matches.
[09:17] So if we do if match objects, which would tell us if there's a match Equals match of every two months equals match ups
[09:35] And at this point we then decide to do print. And if we run that Now there's a chance that the date isn't in the file name.
[09:50] didn't give us the date that. So we need to add analysis and we need to find a way of getting it from the date. So working out what the date is going to be for the file.
[10:04] I Googled it and looked at it up on StackOverflow. it finds the created date, but the reason it gets a bit complicated is because
[10:18] the created date is going to change depending on what platform you're on. If you're not Windows, you might need to use like the modified date. bit of code that I found from somewhere else.
[10:33] So we need to import platform, we need to import date time. So if we do that at the top of the import platform and import date time, we can then use this bit of code.
[10:49] to see whether we on Windows, if we are, we can use get C time. But the reason we can't use this on Mac or Linux is because
[11:02] this will actually give us the modified that if we are on Mac and Windows we can get the timestamp for the birth time to the created date from this OS. But this again only works on Mac or Linux.
[11:16] to use the time that it was last modified, which also is less accurate. you can actually error on Linux because this just doesn't exist on the next.
[11:28] on that timestamp to turn it into the format we want. So here we've got a year, months, day and then we'll see our minutes and seconds.
[11:40] So we're going to do State Created, we're going to call our new method which is called creation date. And here we basically have to pass in the whole pass as a file.
[11:54] name, we need to include folder here as well. Plus we need to include a forward slash and then we need to do the same thing we had up here.
[12:08] So where we were taking the date from the file name, this time we're taking the date from the created date, but we still need that year and months. I'm going to change that file to that created
[12:22] and then we can do the same thing and take the year and months out there as well. Now if we print those, So you can see here
[12:34] Okay, so now we know that this bit works. and we're going to add in a return statement. now to do this, there's a number of ways we could create an object,
[12:49] So if we do return and then we're going to create dictionary with Yeah, month.
[13:06] And then to check this works, we can just add in a print statement, we can do a date another one to print out the month
[13:19] unless you give us a similar output to the one we just had. So we have it as a variable
[13:31] And we've got the one thing I didn't add was up here, we obviously have a Matches now is unlikely that a file wouldn't
[13:44] have occurred to date of any kind, but it's worth doing a check anyway. and just to a check to see, we can just put this inside our statement
[13:56] So what we can do is ready to return zero. this is a string as the other ones are going to be returned to strings. to say, unable to get that,
[14:13] and then we can include a name file that have the issue. So next, now that we've got the year, a month And if it doesn't, we need to try to set
[14:27] whether we've got zeros at all just so we don't continue, So we'll just click here. Equals zero or month equals zero.
[14:44] There's no need to print anything, as we said So assuming we've got a date, We need to then get the folder and create it.
[15:00] So as I showed earlier, to be able to have all of the folders in alphabetical order We need to create a folder name with the month number and the month name that. So I'm going to copy and paste this from what I did earlier.
[15:15] Just so the indentation, at least until I think version 3.10 twice and didn't have any switch statements. to do what we wanted to do.
[15:31] libraries that are in Python, but this is the simplest way of doing it. So they're in alphabetical order when we sort them
[15:43] So we go back down to our code, Yeah. And month. I know we're going to print this folder and see what it gives us and it shit
[15:58] So that's the folder structure we're going to have so we can have 2022 with the August and then set temperature in October. So we we've got our target at the top here, which is Dropbox photos.
[16:14] And we need to check first to see whether it exists. I'm going to do Target Plus.
[16:27] class for 49 and then we can use a Python library to see where this folder exists.
[16:39] So if it doesn't exist or do not, then we want to create it. We can do our stop path that exists which means if it doesn't exist, we need to create it again.
[16:55] That is a one could make does. So you can see here make the directory just to target folder. if it doesn't already exist, ready for our photos to go in.
[17:10] the file that we want to move across or it exists again. If not, oh, stop path still exists.
[17:23] using our target folder, we need to create a target file. file, this is going to be of the target folder
[17:40] plus our separator and then plus our file and then we can check whether it exists or not. So there is another utility, another that we can use here,
[17:56] so we can import the assets util library that allows us to move files quite easily. So if we do move, we then basically want to move it from our source, our source file move to our target fall
[18:09] We need our source file to kind of be source plus file. So that's going to be moving them if they don't already exist
[18:23] And if they do exist, we want to check the size of the file as well and see So so I don't bore you with how we do this. Click to style and we'll put this in here so you can see what it does here.
[18:41] We need to check whether it would exist. It's just so out. for the indentation. So if you have a copy and paste anything
[18:55] from anywhere else, you need to make sure that the indentation is correct. So here we're using again the stats on the file as an SD size.
[19:08] like the source file that we specified here and the target file. and then we can basically remove the source file. The last thing is if the forward exists, but it's a different size in this case.
[19:23] We could rename the file to allow us to move it. occurrence, there's not really any point in doing that to finally, if we run our script, we should find that it moves all files, that we go.
[19:36] we can now see that it's back to having everything in the correct order. the last thing we need to do is have it run automatically. If you're on windows, you want to use task scheduler,
[19:51] but if you're on Linux or Mac, you can use Chrome. So Cron is built in on a system we have to do is to Chrome tab minus E. on the zeroth minute if you want to create your own ones.
[20:07] Quantum dot Guru and I'll put a website down below here. There you can see I've got 0 minutes and this is going to run at minute zero,
[20:19] and then the path through our file and that's all there is to it. to moving all my photos across to the file photo.
[20:31] there's a great book by Al Stiglitz called How to Automate and I'll put a link in the description below for you to try out. if you want to learn a bit more about it and the different branching strategies.
[20:46] Thank you for watching and don't forget to subscribe.
