How to Build a Complete Python Package Step-by-Step
Vložit
- čas přidán 24. 07. 2024
- Do you want to know how to build a complete Python package? Well, look no further! This video will take you step-by-step through the entire process, from creating your project to publishing it on PyPI.
The most popular software licenses available: tldrlegal.com/ or choosealicense.com/
Check the available classifiers here pypi.org/classifiers
Link to Git repository: github.com/ArjanCodes/2023-pa...
✍🏻 Take a quiz on this topic: www.learntail.com/quiz/vgrbvz
🚀 Next-Level Python Skillshare Class: skl.sh/3ZQkUEN
👷 Join the FREE Code Diagnosis Workshop to help you review code more effectively using my 3-Factor Diagnosis Framework: www.arjancodes.com/diagnosis
💻 ArjanCodes Blog: www.arjancodes.com/blog
🎓 Courses:
The Software Designer Mindset: www.arjancodes.com/mindset
The Software Designer Mindset Team Packages: www.arjancodes.com/sas
The Software Architect Mindset: Pre-register now! www.arjancodes.com/architect
Next Level Python: Become a Python Expert: www.arjancodes.com/next-level...
The 30-Day Design Challenge: www.arjancodes.com/30ddc
🛒 GEAR & RECOMMENDED BOOKS: kit.co/arjancodes.
👍 If you enjoyed this content, give this video a like. If you want to watch more of my upcoming videos, consider subscribing to my channel!
💬 Discord: discord.arjan.codes
🐦Twitter: / arjancodes
🌍LinkedIn: / arjancodes
🕵Facebook: / arjancodes
📱Instagram: / arjancodes
♪ Tiktok: / arjancodes
👀 Code reviewers:
- Yoriz
- Ryan Laursen
- James Dooley
- Dale Hagglund
🎥 Video edited by Mark Bacskai: / bacskaimark
💻 Code example by Konstantinos Mastorakis: / konst-mastorakis
🔖 Chapters:
0:00 Intro
1:10 Code example
3:18 Example use cases
4:31 Setuptools package
7:31 Presenting setup.py
10:38 Building and installing a package
12:27 Introducing PyPI
13:54 Publishing the package on PyPI
16:13 Retrospective
19:57 Outro
#arjancodes #softwaredesign #python
DISCLAIMER - The links in this description might be affiliate links. If you purchase a product or service through one of those links, I may receive a small commission. There is no additional charge to you. Thanks for supporting my channel so I can continue to provide you with free content each week!
👷 Join the FREE Code Diagnosis Workshop to help you review code more effectively using my 3-Factor Diagnosis Framework: www.arjancodes.com/diagnosis
Man, you are taking Python learning to the next level. I am a big fan of your videos.
Thank you so much!
I have been learning so much from you. You Not just python but how to gain job ready skills. I used to be afraid that I do not know how to use any of the python I learned. You bring this down to earth with clear succinct and byte sized lessons. THANK YOU! 🙏
Thank you Arjan! It's so valuable to have all of this concentrated into a single video.
Glad it was helpful!
This is amazing information. I believe this will be a great addition( having a course in details) to one of your amazing course library.
Thanks for the complete guide Arjan!
thanks so much Arjan! Stars aligned, I had put knowledge about Python packaging on my to-do list this week haha. All the best!
Wow, glad I could help!
I personally love poetry :) Never learnt the other ways of code packing - so this is neat to know! Thanks.
Packaging is by far the most challenging aspect of Python for me. This helped quite a lot though, thanks
Wanna learn framework building process. Thanks for this awesome tutorial ❤️
Thanks Arjan! Have you ever used declarative metadata? What are their (dis)advantages compared to setuptools?
Great outline. I'm building some common modules at work and this will allow me to easily share with my colleagues
Great video! A good follow up would be good to configure the package with `setup.cfg` or the `.toml` file. The community it's trying to move away from `setup.py` because of security concerns about arbitrary code execution 👍
Sorry for the typos, CZcams won't let me edit the comment 😥
Absolutely! In fact, I just recorded a Poetry video that I’ll publish in a few weeks. I’m also going to move all the code examples I publish to use a toml file from now on.
@@ArjanCodes any plans for (Ana)Conda?
Excellent tutorial as always. Do you have any plans for a tutorial to distribute a python application for win/mac/linux (not just a single package)? Ideally through some kind of automated github pipeline.
Finally, a simple and straightforward approach to do it. Thanks for sharing!
Glad you found it helpful!
Excellent video, Man. keep it up!
I'm eager to watch a video you explain about meta class and meta-programming!
Noted!
Rien a dire si ce n'est MERCI.Tres satisfaisant vos contenus.👍
Thanks you very much for your video, it's very hard to find a clear tutorial about building a python package. Now I'm ready to build a project for server-side transaction verification for mobile 😁
That video was superb. I was so intersting that I forgot to like it in the first place. full of useful information and pro-tips. I just returned to like and leave a comment. Thanks a million.
Thank you and I'm so glad you enjoyed it!
Great video talking about the built in abilities of Python packaging! conda is a neat option too!
Thank you!
Yay, a code roast! XD
There is one big thing that I think I would still do: turn this into a package. Currently, you just have a collection of modules, but you have a proper package. It’s super easy to do with poetry. Why?
• Better separation between the library code (in src/my_package) and the main script (main.py lives outside).
• But instead of a script, you can also create a simple CLI, and for that, having a package will greatly help too
• Right now, running the main application and the tests ONLY WORKS because you run them from the root folder. But imports in the tests would fail if you were to run them from inside the test folder. And you cannot move the main.py elsewhere. Having a package removes those limitations, as you would import from the package in the tests and the scripts. The imports will always work because you have installed the package in your environment, so python knows where to look.
• And as a bonus, this would FIX your red-lines issues in your IDE! I guess what happens is that Pylint is having trouble locations the modules you import. Usually imports work a lot better with packages. BTW, if you have a poetry environment, you have to tell VSCode to use it (cmd palette > Select Python Environment)
I hope I convinced you 😊
Also, one more thing FYI: you have OUTPUT_TYPE = “visualizer” # “visualizer” | ”console”. Right now, the type of this variable would be inferred as string, but if you want to more precisely define it, you can do it like this: “OUTPUT_TYPE: Literal[“visualizer”] | Literal[“console”] = “visualizer”, and Literal imported from typing. However, I’m not sure I really recommend this approach, I wonder if a StrEnum wouldn’t be better…
Okay, thanks for reading if you made it here 😊 And thank you again for the video 😊
Excellent video!
I think there is way too few videos on the topic of packaging python files. So I wouldn't be surprised if this video will generate quite some views...
very helpful video, thank you!
Thank you for the assistance. I found it incredibly helpful. Suppose I have developed an application that is functioning perfectly on my machine. Now, I would like to obtain comprehensive information about the versions of all the requirements and ensure their accuracy. To achieve this, I used the "pip freeze" command to retrieve the versions of the libraries I utilized. I would appreciate your confirmation on whether this is considered a correct practice or if there are any alternative methods you would recommend.
@11:00 I like to install the package like this: pip install -e .
The e flag makes it "editable" so while you continue developing any changes you make will be picked up.
Thanks for the quality video. Thanks to you I have become a much more structured dev.
Glad the videos have been helpful!
Thanks. just what I needed. I am a relative newbie. I am curious about Wade Curry's questions
Hi Arjan, again a great video that covers a topic from begin to end.
Although I have made quite some packages within the company where I work, I've never taken the threshold to actually publish a package (to a company internal Pypi server that is).
I join suggestions from other commenters: could you also do a video on packaging with conda and one with poetry? And what the 'deeper thoughts' behind these alternatives are? So as to understand why they were created anyway? And, if possible, explain why you would favour one over the other? I realize, quite some questions but I sincerely think many struggle with this.
Thanks for sharing! Video on Poetry is coming soon! Stay tuned :)
During the installation part is usefull to use pip install -e . so you don't need to reinstall you package for every modification you do in the project.
@arjancodes please make a video on poetry with typer. Making a CLI and packaging it please
With the test folder being on the same level as your src folder, does that mean you have to build and install your package every time you test? I can't get relative imports to work...
Amazing information 👍👍🙏
Quite good. Thank you!
Glad you liked it, David!
Hmm, I didn't realize that import supported a tuple as the import targets. Not sure how I feel about that, stylistically! Great video though. I wish I'd seen it last year when I was detangling some package creation for a legacy project. Maybe I should try "import time_travel" and help Past-Me....
Excellent tutorial, would you be interested in doing the same but with the poetry library?
Thank you so much
You're most welcome
Hi Arjan, thanks for the video. After I run the build commands I'm not being able to import it. It says that the module was not found. Do you know whyt is it happening? Thanks!
One thing I struggle with python packaging is that most of the programs i develop require additional programs that are not python packages.
How do we make it so that we can make thise requirements explicit and stop a package from instaling and declare the tools missing?
Great video, Arjan. Could you please make a video on Poetry?
Video on Poetry is coming soon! Stay tuned :)
@@ArjanCodes Eagerly waiting for it :)
Any suggestions how to build sip-4.16.5 ? Should MS Visual Studio Express 2010 or mingw/gcc be used?
For example, Nuitka reports:
Python: 3.4.4 (v3.4.4:737efcadf5a6, Dec 20 2015, 19:28:18) [MSC v.1600 32 bit (Intel)]
Version C compiler: C:\Users\Temp\AppData\Local\Nuitka\Nuitka\Cache\downloads\gcc\x86\11.3.0-14.0.3-10.0.0-msvcrt-r3\mingw32\bin\gcc.exe (gcc).
Which seems like both Microsoft Visual Studio 2010 and gcc... so I'd like to build using that gcc rather than microsoft studio since that may mess up Nuitka?
Sir i am having a problem i uploaded the file but when i load the file by pip install none of the function work it only imports nothing works idk why i imported all the modules in the __init__ file can you please help em
What pypi server do you use for private packages?
Why have the app directory? is this the same thing as a src directory?
thanks man😊
Welcome 😊
Not all punctuation are allowed, how select the desired ones
now we want a video about licenses!
Noted!
The tendency nowadays is to avoid packaging as code. People get really imaginative when having the possibility to code and start adding logic in `setup.py`, making the building process not so strait forward.
The first approach was to do all the same but in `setup.cfg`, an INI-like metadata file format that setuptools understands. Severing the possibility to write complex functions that run during the packaging process.
Finally, the consensus is to use a standard build system, that may use setuptools as back-end as well, and define the building rules as metadata too, but in a `pyproject.toml` file.
I think that last approach is the most accurate way to teach Python Packaging today.
A follow up video perhaps, just to show the current recommended way to package? 😅😛
Coming soon! I'm also going to include a pyproject.toml file from now on in my code examples on GitHub :).
You simply goated
Thank you!
amazing topic and video. want to see more on this.
Thank you! Will do!
dude. you literally have a video for every python obstacle I encounter! lol
Haha, happy to be of service 😁
I believed that the "best practice" way to build a wheel is the "pip wheel" command and requirements should be in the usual `requirements.txt` file(s) and be loaded through `pkg_resources.parse_requirements`
Wish you explain package folder structure in more detail.
Can we have an update (if applicable) with regards of python 3.12 PEP 632?
PEP 632 is about distutils-> setuptools migrations, which is probably interesting only to people who have been packaging python software for at least 10 years and do not need tutorials?
Thx for the video, but take into account that according to setuptools docs running python setup.py directly as a script on recent versions is considered deprecated
Woah! Holy crap you look like John Green! Great Videos btw.
Glad you liked the content!
could not do on linkedin - will do on facebook though
where is the version of the package stored?
Can you do build package with poetry too
what about using poetry?
Video on Poetry is coming soon! Stay tuned :)
A M A Z I N G !!
:D
In 2023 you talk about the old setup? I thought it just got removed and we now have toml parser in core. Nevertheless: please more packaging videos. Poetry? Flit?
Video on Poetry is coming soon! Stay tuned :)
have you published it yet
from my perspective the community is moving away from setup tools. I'd probably go over a framework like poetry or flit these days
Are you spying on me?
Everytime you release a video it is relevant to what I am currently working on. xD
But is there a reason why you do not use the folder structure they use on docs.python when they talk about file/folder structure?
(A src folder with the package folder inside)
The same thing happens with me. One day I was searching for pydantic models in python and the same video came in 2 days.
I would like to know too.
give me file which should work as same as the build file works in nodejs
can you give us a video to make a deployment file so that people can access it
With poetry, you can just do "poetry new your_project_name", "poetry build", "poetry publish" and you're done.
Checkout pdm
C'mmon where's poetry?
Coming soon!
I could watch another 12 movies on this topic. It’s the most confusing least intuitive part of python
Next level: Try to compile a Python package written in C with manylinux tools and publish it via an automated GitHub workflow 😂😂😂
I mean the way you publish doesn't really require PyPI. You'd probably be better off with a Git submodule, tbh. Just clone the repo, cd into it and run "pip install .", basically does the same but without PyPI in your way 😉
5:30 - `python setup.py bdist_wheel` apparently does nothing. Did you forget to show how to prepare setup.py?
Even after going ahead and preparing `setup.py` and `__init__.py`, after installation, none of the projects, nor any python CLI can see anything inside the package. Can't import anything, can't use any classes, nothing.
Can you read my mind Arjan cos I was thinking of building a package that incorporates Twitter api with chat gpt. By the way I need frontend volunteers, comment below if you feel it’s a suitable project to be part of.
Not a frontend expert, but would like to be a part of this.
Awesome! Thanks for sharing this.
Do you also cover Julia language?
wow. it is overcomplicated.
Nope, the answer is poetry
I just recorded a video about Poetry, so I agree now :).
The actual answer is pdm! Another example of the „There are 14 competing standards“ xkcd^^ I like poetry but imo pdm is even better.
@@joelbeck1822 totally agree on PDM. poetry capped versions are far more troublesome than useful.
Pipenv is good enough!
I like pbr.
is he dutch? he looks dutch
He is as Dutch as they come.
Just as a complete aside, "PIN number" is repetitive. If you expand it out, you're saying "personal identification number number." The docstring for that function actually gets this right ;)
`:%s/generate_pin_number/generate_pin/g`