6 mins read

You wrote a new Python package that solves a specific problem and it’s now time to share it with the wider Python community. To do so, you need to upload the package to a central repository that can be accessed by developers across the globe.

In today’s article, we are going to discuss how PyPI lets developers share packages with other people who may wish to use that particular functionality in their own application. Additionally, we are going to introduce a step-by-step guide to help you upload your Python package on PyPi so that it is available to every Python user. I’ll use a real end-to-end example so that all the steps are crystal clear.

What is PyPI

The Python Package Index, abbreviated as PyPI, is the official repository of software for the Python programming language. By default, pip — which is the most popular Python package manager — uses PyPI as the source for retrieving package dependencies.

PyPI lets you find, install and even publish your Python packages so that they are widely available to the public. More than 300,000 different packages are currently published in the index with more than 2,500,000 releases being distributed to users.

How to make your Python package available on PyPi

In the following section, we’ll explore the steps you need to follow in order to publish your Python package on PyPI and make it available on pip.

Step 1: Ensure you have pip installed

If you are using Python 2 ≥ 2.7.9 or Python 3 ≥ 3.4 pip should be already installed. But if for any reason you may need to install it simply run the commands below (instructions are suitable for Unix):

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python get-pip.py

And that’s it! You can ensure that pip is now installed as below.

$ python -m pip --version
pip X.Y.Z from .../site-packages/pip (python X.Y)

Step 3: Package your Python code

I created a sample project on GitHub so that it’s easier to demonstrate how to publish your own packages on PyPI. The project can be found on this link — normally you’ll need to create at least the files below:

• README.rst: It is highly recommended to include a README file where you should outline the basic functionality offered by your package. Additionally, you may also include installation instructions or a usage guide.
• LICENSE.txt: It’s always best to include a license in the package you intend to make it widely available. For more options and details refer to the Licensing a repository section of GitHub docs.
• setup.py: This file should be placed into the top-level directory of your project structure. In this file, you can specify configurations for your Python project. For more details, you can refer to the official documentation. The setup.py file for our example project can be found on GitHub and is identical to the one shown below.

from setuptools import setup, find_packages

setup(
    name='example_publish_pypi_medium',
    version='0.6',
    license='MIT',
    author="Giorgos Myrianthous",
    author_email='email@example.com',
    packages=find_packages('src'),
    package_dir={'': 'src'},
    url='https://github.com/gmyrianthous/example-publish-pypi',
    keywords='example project',
    install_requires=[
          'scikit-learn',
      ],

)

• setup.cfg: This file contains default options for setup.py commands.

[metadata]
description-file=README.md
license_files=LICENSE.rst

The overall structure of our example project is shown below

.
├── LICENSE.txt
├── README.rst
├── setup.cfg
├── setup.py
├── src
│   ├── example_publish_pypi_medium
│   │   ├── __init__.py
│   │   └── example
│   │       ├── __init__.py
│   │       └── custom_sklearn.py
├── tests
│   ├── __init__.py
│   └── example
│       ├── __init__.py
│       └── test_custom_sklearn.py

Now we just need to publish our package on PyPI so that other users can install it on their local machines. Note that our example package depends on scikit-learn and this dependency is explicitly specified in setup.py file. Therefore, when users install your package using say pip, the specified dependencies will also be installed.

Step 3: Create the source distribution of the package

Now that our source code is structured and contains all the required files for packaging we can go ahead and create the source distribution.

A source distribution — commonly referred to as sdist — is a distribution that contains the setup.py file along with the source code and data files (as specified in the setup.py and/or setup.cfg)

You can create the source distribution of the package by running the command given below:

python setup.py sdist

If everything goes to plan should create a .tar.gz file under the newly created directory dist. In my case setuptools also added a MANIFEST file too.

Note: If by any chance you’ve got a warning similar to

Unknown distribution option: 'install_requires'

you can ignore it for now.

Step 4: Install twine

twine is a utility package that is used for publishing Python packages on PyPI.

pip install twine

Step 5: Create a PyPI Account

In order to publish a package on PyPI you have to create an account. You can do so by visiting this link. It’s completely free and to sign up you just need to provide your e-mail address, a username, and password.

Step 6: Upload the source distribution on PyPI

Finally, we’ll now use twine in order to upload the created source distribution on PyPI.

twine upload dist/*

You will be prompted to type your username and password and your package will be finally made available on PyPI:

twine upload dist/*
Uploading distributions to https://upload.pypi.org/legacy/
Enter your username: YOUR_USER_NAME
Enter your password: YOUR_PASSWORD
Uploading example_publish_pypi_medium-0.6.tar.gz
100%|██████████████████████████████████████████████████████████████| 3.49k/3.49k [00:01<00:00, 2.17kB/s]View at:
https://pypi.org/project/example-publish-pypi-medium/0.6/

Voila! Our example package is now available on PyPI.

Installing the published package with pip

Now let’s verify that our package works as expected.

As we already mentioned, the default source index of pip is PyPI. Therefore, our package can be installed directly from pip. To do so, run

pip install example-publish-pypi-medium==0.6

The package should be successfully installed on your local machine. If you inspect the output you’ll also see that scikit-learn is also installed since it is a dependency that we also specified while we were packaging our source code.

Finally, we can verify that the package works as expected:

$ python3
>>> from example_publish_pypi_medium.example import custom_sklearn>>> custom_sklearn.get_sklearn_version()
'0.24.1'

Final Thoughts

It’s important to know how you can distribute your Python packages and make them widely available. In today’s article, we discussed how PyPI is used as a central repository containing Python packages. Additionally, we’ve seen how to package and publish your own Python package to PyPI so that it is widely available.

Note however that it is highly recommended to use TestPyPI, which is a separate test instance of the original index that allows users to try out distribution tools and processes without affecting the real index. Once you verify that everything behaves as expected you can then publish it to the real PyPI using the commands we explored earlier.

References:

https://towardsdatascience.com/how-to-upload-your-python-package-to-pypi-de1b363a1b3

https://python-packaging.readthedocs.io/en/latest/minimal.html

https://medium.com/@joel.barmettler/how-to-upload-your-python-package-to-pypi-65edc5fe9c56

https://realpython.com/pypi-publish-python-package/

Python Packaging from Init to Deploy