Home

Aaron Tan's Blog

14 May 2018 ∞

Creating an executable with PyQt5, PyInstaller, and more

While I was working on the Quaver Lyrics Finder project (it’s my first time writing Python, please be gentle), I wanted to make it into an easy-to-use program that most people to use. That meant:

  1. Looking native, presenting a GUI when clicked.
  2. Turning it into a self-contained executable file that does not rely on having Python or other dependencies being installed.
  3. Creating a file that users could download in an expected file format. This means having a .exe or .msi for Windows, a .dmg for macOS, and a .deb for Ubuntu.

Here’s what I did.

This is the first part of the series where I write about things I learned while writing a desktop app. The second post can be found here.

Contents

  1. Looking native
  2. Turning the project into a self-contained executable file
  3. Creating a package in an expected file format
  4. Closing Notes

1. Looking native

For creating a GUI in Python, I chose to use PyQt5, a framework that provides Python bindings for the great C++ Qt framework.

Reasons in favour of PyQt:

  • Looks like native elements.
  • Compatibility with Python 3.5+.
  • Well tested and documented.

Something that you should consider about PyQt5 is that licensed under the GPL 3.0 license. That means that if you distribute the Qt Gui Toolkit as part of your application binary, your program must also be licensed under a GPL-compatible license. Since my project was open-source anyway and uses the MIT license, this was fine by me.

Other possible choices:

  • Pyside: Another framework for bindings for Qt. In favour: Less restrictive licensing (LGPL instead of GPL). Against: Does not support Qt5, a little bit less well-documented because it’s less well funded than PyQt.
  • TkInter: A GUI framework that comes pre-installed with Python. In favour: Easy to set up, since it’s pre-installed with Python. Against: Does not look native, less popular than PyQt/PySide.
  • Kivy: Another GUI framework. Against: Not Qt, but seems decent otherwise.
  • wxPython: Doesn’t support Python 3, so this was out.

I’ve used PyQt on four different platforms - macOS, Windows, Ubuntu, and elementary OS. The GUI looks pretty good on all of them, so the cross-platform claim checks out!

2. Turning the project into a self-contained executable file

I’m using PyInstaller to generate an executable for my python files.

PyInstaller:

  • Checks for dependencies for all files, and puts them into your executable.
  • Works cross-platform: same commands in macOS, Windows, Linux.
  • Is easy to use - one command in whatever shell you prefer.
  • Is actively maintained, has decent documentation, and relatively popular.

Since it’s popular, chances are that when you have a problem, there’s somebody else who’s already run into it. Stack Overflow, the GitHub page, and even just the documentation are life-savers. Keep in mind also that PyInstaller is not a cross-compiler, i.e. you can’t build Windows .exe files when using macOS, etc. Use PyInstaller in a VM to do that.

Other possibilities include:

  • bbfreeze: But it’s unmaintained and does not support Python 3.
  • py2app: It’s not being maintained, and not cross-platform (you need to use py2exe to create an executable for Windows).
  • cx_Freeze: Less actively maintained compared to PyInstaller.
  • pyqtdeploy: More fussy than PyInstaller, and harder to set up.

2.1 Using PyInstaller

PyInstaller can be downloaded with pip:

pip install pyinstaller

Then, starting off with PyInstaller should be as simple as running:

pyinstaller <your_file>.py

It checks for dependencies, analyzes <your_file>.py, and generates an executable file in dist/ as well as a <your_file>.spec file in the same directory as <your_file>.py. If all goes according to plan, there should now be a single executable file that you can click to open in dist/ and it should run as if you had run your script from the command line.

For more details, read their docs.

2.1.1 Useful options

--onefile allows you to create a single executable.

--onedir creates executables in a folder structure. Useful for debugging.

pyinstaller <your_file>.spec will use the specified spec file to build the executable. This is very useful if you’ve customized the spec file.

2.2 Customizing the spec file

The spec file is a Python script that PyInstaller uses to determine what to build and where to build it. Understanding this is essential to understanding PyInstaller. There are several customizations that I’ve done to my spec files that have come in handy.

A spec file is generated automatically in the same directory as your file when you run pyinstaller <your_file>py. You can customize it from there on. Note that if you run pyinstaller <your_file.py again, it will generate another spec file that overwrites any customizations that you made! To avoid this, give pyinstaller a spec file as an argument, e.g. pyinstaller <your_file>.spec.

2.2.1 Assets

If you refer to assets by a relative path in Python file <your_file>.py, likeQtGui.QIcon('./assets/icon.png'),the executable that PyInstaller generates won’t include them in the correct place, and your program will not be able to find the assets. This happens for images, text files, data files, and more.

The workaround that I’ve found:

  1. Add these lines after the a = Analysis(...) code block in the spec file, with more entries as appropriate: 

    a.datas += [('<actual_path_to_file_1>', '<path_used_to_refer_to_asset_in_code_1>', 'DATA'),
         ('<actual_path_to_file_2>', '<path_used_to_refer_to_asset_in_code_2>', 'DATA')]

    Note that in certain filesystems, case matters! This bit me in Ubuntu.

    The path to the file is always relative to the directory where you the .py file you are asking PyInstaller to analyse is. As an example, this is what the directory structure could look like:

    _ my_folder
|
├─── my_file.py
│
└───_assets
    |
    ├─── icon.png
    │
    └─── settings.ini

    With that directory structure, the code snippet would become:

    a.datas += [('./assets/icon.png', './assets/icon.png', 'DATA'),
         ('./assets/settings.ini', './assets/settings.ini', 'DATA')]


  2. After having modified your spec file, you’ll need to go back to <your_file>.py and add the following code somewhere: 

    import os, sys
# Translate asset paths to useable format for PyInstaller
def resource_path(relative_path):
  if hasattr(sys, '_MEIPASS'):
      return os.path.join(sys._MEIPASS, relative_path)
  return os.path.join(os.path.abspath('.'), relative_path)

    Explanation: PyInstaller creates a temporary folder called _MEIPASS, that contains the files as named in a.datas. Thus, the _MEIPASS folder exists only when the code is running as an executable bundle generated by PyInstaller. This function checks for that folder, and if it exists, thus knows that it’s running the executable. When the code is in an executable bundle, this function will convert your relative path to the path in the _MEIPASS folder, and otherwise leaves it alone.

  3. Once you’ve added the function to your code, you’ll need to update every place that has a relative path to call that function.

    As an example: 
    # Replace QtGui.QIcon('./assets/icon.png') with:
QtGui.QIcon(resource_path('./assets/icon.png'))


  4. And finally, run PyInstaller with your new spec file. For example, when I’m generating a bundle for Quaver, I do pyinstaller quaver-onefile.spec.

2.2.2 Generating a bundle for macOS

PyInstaller is really good for creating bundles that conform to Apple’s Bundle Structure, making them easily code-signable. Here’s what I add to my spec file after the pyz code block to generate a bundle for macOS.

# First, generate an executable file
# Notice that the icon is a .icns file - Apple's icon format
# Also note that console=True
if sys.platform == 'darwin':
  exe = EXE(pyz,
            a.scripts,
            a.binaries,
            a.zipfiles,
            a.datas,
            name='Quaver',
            debug=False,
            strip=False,
            upx=True,
            runtime_tmpdir=None,
            console=True,
            icon='assets/icon.icns')

# Package the executable file into .app if on OS X
if sys.platform == 'darwin':
   app = BUNDLE(exe,
                name='Quaver.app',
                info_plist={
                  'NSHighResolutionCapable': 'True'
                },
                icon='assets/icon.icns')

In BUNDLE(), info_plist will create the appropriate <key> and <string> in the .app bundle structure. For example, the NSHighResolutionCapable example looks like this in the Info.plist file in Quaver.app/Contents/.

...
<key>NSHighResolutionCapable</key>
<string>True</string>
...

Unfortunately, specifying the types of documents the app supports doesn’t work with this format. I have a post about this here, but you need a CFBundleDocumentTypes key with an array of values, which is too complicated for PyInstaller’s spec file to handle. The only solution is to create and save a separate Info.plist file, then replace the one PyInstaller generates in your_app.app/Contents/. See here for more details. :/

2.2.3 Generating a .exe for Windows and executable for Linux

Here’s what I’m using in my spec file for Windows and Linux:

# Generate an executable file
# Notice that the icon is a .ico file, unlike macOS
# Also note that console=False
if sys.platform == 'win32' or sys.platform == 'win64' or sys.platform == 'linux':
  exe = EXE(pyz,
            a.scripts,
            a.binaries,
            a.zipfiles,
            a.datas,
            name='Quaver',
            debug=False,
            strip=False,
            upx=True,
            runtime_tmpdir=None,
            console=False,
            icon='assets/icon.ico')

3. Creating a package in an expected file format

After customizing the spec file and running PyInstaller, we now have a nice executable file. A .app on macOS, a .exe on Windows, and an executable file on Linux. The next step is to package it into something that users are used to downloading.

3.1 Creating a .dmg for macOS

I’m not fussy, so I used a simple command line tool that creates a .dmg from a .app, called create-dmg. It’s purposely simple, and doesn’t offer many options.

Assuming you have npm installed (otherwise, brew install npm (and if you don’t have brew click here):

sudo npm install -g create-dmg

# If you get an error about "EACCES: permission denied,
# mkdir '/usr/local/lib/node_modules/create-dmg/node_modules/fs-xattr/build'", 
# run this command instead and see 
# https://github.com/npm/npm/issues/17268#issuecomment-310167614 for more details
sudo npm install -g create-dmg --unsafe-perm=true --allow-root

Now, cd into the directory containing your .app file generated by PyInstaller, and run:

create-dmg '<your_app>.app'

# If you gt an XCode error like "gyp: No Xcode or CLT version detected!"
# try running the command below then retry the above command.
# See https://stackoverflow.com/questions/27665426/trying-to-install-bcrypt-into-node-project-node-set-up-issues
# for more information
sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer/

You should now have a new <your_app_name> <sem_ver>.dmg file in the directory where you ran create-dmg! Create-dmg pulls the version number from your app’s Contents/Info.plist folder, so if it’s wrong, update the Info.plist file first.

3.2 Creating a .msi file for Windows

If you don’t need to change the registry, you could just distribute the .exe file that PyInstaller generated, since the .exe file is basically a portable version of your program. Otherwise, you’ll need to create a separate installer executable to set things up correctly. Another benefit of an msi file is that it can properly integrate with the start menu and create desktop shortcuts easily.

I’m using the free version of Advanced Installer to do this. Some gotchas: make sure you create a “Simple” project instead of a “Professional” or anything else, because “Simple” is the only one that’s free, and you cannot convert one type of project to another later on. Advanced Installer lets you create Professional projects when you download by automatically activating a trial, which is sneaky.

Customize as necessary. See their website for a simple tutorial on how to get started and their professional guide which walks through a few more tools.

Other competitors:

  • WiX Toolset:
    • ++ Immensely powerful and more customizable.
    • ++ Uses text files for configuration, which simplifies version control and portability.
    • ++ Free and open source.
    • ++ Well documented, and tested.
    • -- Significantly harder to learn and get started with. Since my project didn’t have big requirements (all I wanted to do was add a few registry keys), learning to use WiX would have been overkill.
  • InstallShield:
    • ++ Powerful, solid, and well-liked.
    • ++ Good GUI.
    • ++ Good support.
    • ---- Literally costs thousands of dollars. I’m not exactly going to spend that much for a one-time project.

3.3 Creating a .deb file for Ubuntu

Unlike macOS and Windows, the tools to create a .deb file are built-in to the operating system!

You’ll need to create a directory structure as follows:

_<project>_<major version>.<minor version>-<package revision>
├───_DEBIAN
│   │
│   └─── control
└───_usr
    |
    ├───_local
    │   |
    │   └───_bin
    │       │
    │       ├───EXECUTABLE_FILE
    │       │
    │       └───<project>.svg
    └───_share
        │
        └───_applications
            │
            └───<project>.desktop

The control file is what the advanced package tool reads to see what needs to be changed and which app is being installed. See more in the documentation. A basic control file might look something like this:

Package: com.aaronhktan.quaver
Version: 0.5-513
Section: base
Priority: optional
Architecture: amd64
Maintainer: Aaron Tan <[email protected]>
Description: Quaver - Lyrics Finder: Quickly find lyrics for your songs with this program!

The <project>.svg file should contain an SVG of your app icon.

The <project>.desktop file allows the launcher to discover your program, and integrates it with your system. See here for more information. A basic .desktop file might look something like this:

[Desktop Entry]
Version=0.5.513
Type=Application
Name=Quaver
GenericName=Lyrics Finder
Comment=Quickly find lyrics for your songs.
Exec=/usr/local/bin/Quaver %F
Terminal=false
MimeType=audio/x-mp3;audio/mp4;audio/x-m4a;video/x-m4v;audio/x-tta;audio/x-ape;audio/x-ms-wma;audio/x-pn-aiff;audio/x-flac;audio/ogg
Icon=/usr/local/bin/quaver.svg
Keywords=Quaver;Audio;Lyrics;Finder;Editor;Songs;
Categories=Audio;Music;Editor;AudioVideo;
StartupNotify=true

Once all that is done, all that you have to do is a simple command, and your .deb file will be created in the same directory as the folder.

dpkg-deb --build <project>_<major version>.<minor version>-<package revision>

4. Closing notes

Writing the functionality isn’t the last thing that needs to be done – just look at this entire blog post as an example. Keep working through it, and it’ll all work out.

If you’re interested in finding out more about the “Open with” functionality and the second post of this series, see here!

aaron at 12:03

footer
‹ ›