Mkvirtualenv pythonanywhere

Now that development teams know about CI/CD, there is no reason for deployments to become a time-consuming and cumbersome process. CI/CD may start with continuous testing, but adding automated deployments takes your CI/CD practice to the next level.

Continuous deployment slashes the time it takes to release so you can spend more time improving the quality of your applications. In this tutorial, I will guide you through using automated continuous deployments to deploy Flask applications to PythonAnywhere. I want you to feel like a “rock-star” level CI/CD practitioner by the time we wrap up the tutorial.

Prerequisites

Before you start, make sure these items are in place:

• A PythonAnywhere hacker plan
• A CircleCI account
• A GitHub account
• Basic knowledge of CI/CD
• Knowledge of API documentation with Swagger UI

Our tutorials are platform-agnostic, but use CircleCI as an example. If you don’t have a CircleCI account, sign up for a free one here.

The sample application I am using for this tutorial is a Flask API. The Flask application exposes a Swagger API that we can use to create and retrieve books. Get the source code of the application by cloning this GitHub repository:

import sys

# add your project directory to the sys.path
project_home = '/home/waweru/automating-flask-deployments-with-pythonanywhere'
if project_home not in sys.path:
    sys.path = [project_home] + sys.path

# import flask app but need to call it "application" for WSGI to work
from run import app as application  

You will use the cloned repository only for PythonAnywhere configuration, which is covered later on in the tutorial.

What is PythonAnywhere?

PythonAnywhere is a hosting platform and an alternative to platforms such as AWS, Azure, Google Cloud, or Heroku. The PythonAnywhere platform provides a cloud-based, easy-to-use tool for development and deployment online.

Automating deployments to PythonAnywhere

Automating continuous deployments to PythonAnywhere offers many benefits for developers:

• Easy and quick to set up
• Built for performance
• Free for open source projects

I have created a flow diagram (using CircleCI as an example) that shows how automating continuous deployments to PythonAnywhere works. The process can be adapted for any CI/CD tool and any other hosting platform.

The diagram shows how to use the CircleCI API with the CircleCI web interface and configuration file to configure deployment jobs. The diagram also shows the process by which all the CircleCI steps and services are executed from a high-level view. Take note of the separation of contexts that span from application cloning to preparing the deployment and then deploying the application.

Setting up PythonAnywhere

PythonAnywhere makes it simple to get started and set up. The PythonAnywhere platform:

• Comes with pre-installed Python libraries
• Lets you set up different environments for an application
• Supports scheduling tasks like regular database clean up or using cron jobs for repetitive tasks

Once you log in to your registered PythonAnywhere hacker account, open the

import sys

# add your project directory to the sys.path
project_home = '/home/waweru/automating-flask-deployments-with-pythonanywhere'
if project_home not in sys.path:
    sys.path = [project_home] + sys.path

# import flask app but need to call it "application" for WSGI to work
from run import app as application  

import sys

# add your project directory to the sys.path
project_home = '/home/waweru/automating-flask-deployments-with-pythonanywhere'
if project_home not in sys.path:
    sys.path = [project_home] + sys.path

# import flask app but need to call it "application" for WSGI to work
from run import app as application  

When setup is complete, you will be redirected to your web application’s dashboard. You can use the dashboard to further configure your application using the console interface.

From the console, you can create a virtual environment for your application using the specific python version declaration version:

import sys

# add your project directory to the sys.path
project_home = '/home/waweru/automating-flask-deployments-with-pythonanywhere'
if project_home not in sys.path:
    sys.path = [project_home] + sys.path

# import flask app but need to call it "application" for WSGI to work
from run import app as application  

mkvirtualenv flask-venv --python=’/usr/bin/python3.9

Cloning the application to the PythonAnywhere console

Next, clone the application from GitHub to the PythonAnywhere console. This is an important step because it also sets up the remote

import sys

# add your project directory to the sys.path
project_home = '/home/waweru/automating-flask-deployments-with-pythonanywhere'
if project_home not in sys.path:
    sys.path = [project_home] + sys.path

# import flask app but need to call it "application" for WSGI to work
from run import app as application  

import sys

# add your project directory to the sys.path
project_home = '/home/waweru/automating-flask-deployments-with-pythonanywhere'
if project_home not in sys.path:
    sys.path = [project_home] + sys.path

# import flask app but need to call it "application" for WSGI to work
from run import app as application  

import sys

# add your project directory to the sys.path
project_home = '/home/waweru/automating-flask-deployments-with-pythonanywhere'
if project_home not in sys.path:
    sys.path = [project_home] + sys.path

# import flask app but need to call it "application" for WSGI to work
from run import app as application  

Installing dependencies

Change directory:

import sys

# add your project directory to the sys.path
project_home = '/home/waweru/automating-flask-deployments-with-pythonanywhere'
if project_home not in sys.path:
    sys.path = [project_home] + sys.path

# import flask app but need to call it "application" for WSGI to work
from run import app as application  

import sys

# add your project directory to the sys.path
project_home = '/home/waweru/automating-flask-deployments-with-pythonanywhere'
if project_home not in sys.path:
    sys.path = [project_home] + sys.path

# import flask app but need to call it "application" for WSGI to work
from run import app as application  

Install all dependencies with:

version: 2.1
orbs:
  python: circleci/[email protected]

workflows:
  build-app-with-test:
    jobs:
      - build-and-test
      - deploy:
          requires:
            - build-and-test
jobs:
  build-and-test:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: |
            pipenv --three
            pipenv install
      - run:
          name: Run tests
          command: pipenv run pytest
  deploy:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Deploy Over SSH
          command: |
            ssh-keyscan -H ssh.pythonanywhere.com >> ~/.ssh/known_hosts
            ssh [email protected]$SSH_HOST "cd automating-flask-deployments-with-pythonanywhere; git pull";

After cloning the Flask application, you need to tell PythonAnywhere where your project is located. To do this, configure the location using the Web tab. Without closing the bash console (on the hamburger menu at the far right), open the Web tab and set the source code path for the project, the virtual environment path, and the static files, including the Swagger UI YAML file.

Set the source code path to:

version: 2.1
orbs:
  python: circleci/[email protected]

workflows:
  build-app-with-test:
    jobs:
      - build-and-test
      - deploy:
          requires:
            - build-and-test
jobs:
  build-and-test:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: |
            pipenv --three
            pipenv install
      - run:
          name: Run tests
          command: pipenv run pytest
  deploy:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Deploy Over SSH
          command: |
            ssh-keyscan -H ssh.pythonanywhere.com >> ~/.ssh/known_hosts
            ssh [email protected]$SSH_HOST "cd automating-flask-deployments-with-pythonanywhere; git pull";

After creating the virtual environment, you can open the console with it. For now you can ignore this option.

Note: To get the working directory path, use the bash

version: 2.1
orbs:
  python: circleci/[email protected]

workflows:
  build-app-with-test:
    jobs:
      - build-and-test
      - deploy:
          requires:
            - build-and-test
jobs:
  build-and-test:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: |
            pipenv --three
            pipenv install
      - run:
          name: Run tests
          command: pipenv run pytest
  deploy:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Deploy Over SSH
          command: |
            ssh-keyscan -H ssh.pythonanywhere.com >> ~/.ssh/known_hosts
            ssh [email protected]$SSH_HOST "cd automating-flask-deployments-with-pythonanywhere; git pull";

Congratulations! Your Flask project has been set up and successfully hosted on PythonAnywhere. Open the link to your web application to review the default PythonAnywhere screen.

Great work so far, but there are still a few more steps to go.

The page for your application displays PythonAnywhere’s default welcome message. To open your Flask application, you need to make a few changes to the PythonAnywhere’s config file.

Open the WSGI configuration file from the dashboard and modify the configuration.

WSGI (Web Server Gateway Interface) is a Python interface to the web server. It allows you to write Python code that can be run on the server and is used to configure the web server. With WSGI you can forward requests from a web server to the Flask backend and from the web server back to the requestor.

To enable execution of your Flask application, remove the

version: 2.1
orbs:
  python: circleci/[email protected]

workflows:
  build-app-with-test:
    jobs:
      - build-and-test
      - deploy:
          requires:
            - build-and-test
jobs:
  build-and-test:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: |
            pipenv --three
            pipenv install
      - run:
          name: Run tests
          command: pipenv run pytest
  deploy:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Deploy Over SSH
          command: |
            ssh-keyscan -H ssh.pythonanywhere.com >> ~/.ssh/known_hosts
            ssh [email protected]$SSH_HOST "cd automating-flask-deployments-with-pythonanywhere; git pull";

Also, configure the path to your project directory and the entry point of your application:

import sys

# add your project directory to the sys.path
project_home = '/home/waweru/automating-flask-deployments-with-pythonanywhere'
if project_home not in sys.path:
    sys.path = [project_home] + sys.path

# import flask app but need to call it "application" for WSGI to work
from run import app as application  

Save the changes and close the file. Next, you need to reload the app and append the suffix

version: 2.1
orbs:
  python: circleci/[email protected]

workflows:
  build-app-with-test:
    jobs:
      - build-and-test
      - deploy:
          requires:
            - build-and-test
jobs:
  build-and-test:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: |
            pipenv --three
            pipenv install
      - run:
          name: Run tests
          command: pipenv run pytest
  deploy:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Deploy Over SSH
          command: |
            ssh-keyscan -H ssh.pythonanywhere.com >> ~/.ssh/known_hosts
            ssh [email protected]$SSH_HOST "cd automating-flask-deployments-with-pythonanywhere; git pull";

You have successfully deployed your Flask application to PythonAnywhere.

Next you will need to configure your application to be automatically deployed by CircleCI every time you make deployments to the main branch.

Setting up CircleCI integration

To set up CI/CD in this project, use

version: 2.1
orbs:
  python: circleci/[email protected]

workflows:
  build-app-with-test:
    jobs:
      - build-and-test
      - deploy:
          requires:
            - build-and-test
jobs:
  build-and-test:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: |
            pipenv --three
            pipenv install
      - run:
          name: Run tests
          command: pipenv run pytest
  deploy:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Deploy Over SSH
          command: |
            ssh-keyscan -H ssh.pythonanywhere.com >> ~/.ssh/known_hosts
            ssh [email protected]$SSH_HOST "cd automating-flask-deployments-with-pythonanywhere; git pull";

Note: SSH also known as Secure Shell or Secure Socket Shell, is a network protocol for operating network services to securely connect over an unsecure network. Applications of ssh include; remote command-line login and remote command execution.

The following diagram shows how CI/CD is applied with CircleCI and PythonAnywhere using ssh.

The illustration shows that when you push your code to GitHub, it kick starts a process to deploy the code to PythonAnywhere. Once the deployment is complete, you can then connect to the PythonAnywhere server and pull the latest code from the main branch. Now that you know how it works, your next step is to write a CircleCI configuration file to achieve it.

Initializing CircleCI on the dashboard

To set up any project in CircleCI, start on the CircleCI dashboard. Go to the Projects section. All the GitHub repositories associated with your GitHub username or organization are listed. For this tutorial, the repository you want to set up is

version: 2.1
orbs:
  python: circleci/[email protected]

workflows:
  build-app-with-test:
    jobs:
      - build-and-test
      - deploy:
          requires:
            - build-and-test
jobs:
  build-and-test:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: |
            pipenv --three
            pipenv install
      - run:
          name: Run tests
          command: pipenv run pytest
  deploy:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Deploy Over SSH
          command: |
            ssh-keyscan -H ssh.pythonanywhere.com >> ~/.ssh/known_hosts
            ssh [email protected]$SSH_HOST "cd automating-flask-deployments-with-pythonanywhere; git pull";

On the Projects dashboard, select the option to set up the selected project. Select the option for using an existing configuration. The first configuration step is complete.

Note: This is not a mandatory step if you have already cloned this repository, but is an important one for you to learn how you would set it up in your own project.

First, create a

version: 2.1
orbs:
  python: circleci/[email protected]

workflows:
  build-app-with-test:
    jobs:
      - build-and-test
      - deploy:
          requires:
            - build-and-test
jobs:
  build-and-test:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: |
            pipenv --three
            pipenv install
      - run:
          name: Run tests
          command: pipenv run pytest
  deploy:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Deploy Over SSH
          command: |
            ssh-keyscan -H ssh.pythonanywhere.com >> ~/.ssh/known_hosts
            ssh [email protected]$SSH_HOST "cd automating-flask-deployments-with-pythonanywhere; git pull";

version: 2.1
orbs:
  python: circleci/[email protected]

workflows:
  build-app-with-test:
    jobs:
      - build-and-test
      - deploy:
          requires:
            - build-and-test
jobs:
  build-and-test:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: |
            pipenv --three
            pipenv install
      - run:
          name: Run tests
          command: pipenv run pytest
  deploy:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Deploy Over SSH
          command: |
            ssh-keyscan -H ssh.pythonanywhere.com >> ~/.ssh/known_hosts
            ssh [email protected]$SSH_HOST "cd automating-flask-deployments-with-pythonanywhere; git pull";

Setting up CircleCI

Your CircleCI configuration file will execute your tests and deploy your application to PythonAnywhere. In the

version: 2.1
orbs:
  python: circleci/[email protected]

workflows:
  build-app-with-test:
    jobs:
      - build-and-test
      - deploy:
          requires:
            - build-and-test
jobs:
  build-and-test:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: |
            pipenv --three
            pipenv install
      - run:
          name: Run tests
          command: pipenv run pytest
  deploy:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Deploy Over SSH
          command: |
            ssh-keyscan -H ssh.pythonanywhere.com >> ~/.ssh/known_hosts
            ssh [email protected]$SSH_HOST "cd automating-flask-deployments-with-pythonanywhere; git pull";

version: 2.1
orbs:
  python: circleci/[email protected]

workflows:
  build-app-with-test:
    jobs:
      - build-and-test
      - deploy:
          requires:
            - build-and-test
jobs:
  build-and-test:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: |
            pipenv --three
            pipenv install
      - run:
          name: Run tests
          command: pipenv run pytest
  deploy:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Deploy Over SSH
          command: |
            ssh-keyscan -H ssh.pythonanywhere.com >> ~/.ssh/known_hosts
            ssh [email protected]$SSH_HOST "cd automating-flask-deployments-with-pythonanywhere; git pull";

In this CircleCI configuration, you are creating two jobs. The first job is a build job responsible for installing the dependencies and running the tests. The deploy job is responsible only for deploying the application to PythonAnywhere. Running the build and test job followed by the deploy job ensures deployment happens only after successful tests.

jobs:
  - build-and-test
  - deploy:
      requires:
        - build-and-test

After the tests pass, you can deploy your application to PythonAnywhere using the deploy job. To allow deployment of the application you need to generate SSH keys in your PythonAnywhere account, and then add that private key to your GitHub account.

Generating SSH keys

You do not want to always use a password to connect to PythonAnywhere. To allow connection to PythonAnywhere without entering a password every time, you can generate SSH keys on the PythonAnywhere console. From the PythonAnywhere console, run this command:

ssh-keygen -t rsa -b 2048

You will be prompted to enter a passphrase for additional security. You can add this or leave it blank.

Great! You have generated your keys. Now you can use them in CircleCI to automatically connect to PythonAnywhere. I will cover that in the next section.

First though, you need PythonAnywhere to know that your public key is an authorized key. Add the public key to

jobs:
  - build-and-test
  - deploy:
      requires:
        - build-and-test

ssh-copy-id [email protected]

You will be prompted to enter your password. Once authorized, you can connect to PythonAnywhere without having to use a password ever again. Cool right? Test by running this command in the PythonAnywhere console:

ssh -i ~/.ssh/id_rsa_pub [email protected]

This command should log you into the PythonAnywhere server from the console without entering a password. Your next step is adding SSH keys to CircleCI.

Adding SSH keys to CircleCI

Following steps in the previous section you were able to generate both the private and public keys for the PythonAnywhere account. Now you can copy your private key to CircleCI.

To avoid adding overhead to the server, you can just display the private key and then copy it to the clipboard. In the PythonAnywhere console, enter this command:

 cat ~/.ssh/id_rsa

Once you have your private key copied to the clipboard, navigate to the CircleCI project settings for your project and add the private key there.

Deploying with CircleCI

The basic setup for deployment to CircleCI is complete. Go back to the

jobs:
  - build-and-test
  - deploy:
      requires:
        - build-and-test

version: 2.1
orbs:
  python: circleci/[email protected]

workflows:
  build-app-with-test:
    jobs:
      - build-and-test
      - deploy:
          requires:
            - build-and-test
jobs:
  build-and-test:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Install dependencies
          command: |
            pipenv --three
            pipenv install
      - run:
          name: Run tests
          command: pipenv run pytest
  deploy:
    docker:
      - image: cimg/python:3.9
    steps:
      - checkout
      - run:
          name: Deploy Over SSH
          command: |
            ssh-keyscan -H ssh.pythonanywhere.com >> ~/.ssh/known_hosts
            ssh [email protected]$SSH_HOST "cd automating-flask-deployments-with-pythonanywhere; git pull";

jobs:
  - build-and-test
  - deploy:
      requires:
        - build-and-test

jobs:
  - build-and-test
  - deploy:
      requires:
        - build-and-test

Then authenticate using SSH and pull the code from GitHub with this command:

ssh [email protected]$SSH_HOST "cd automating-flask-deployments-with-pythonanywhere; git pull";

Note that we are using the

jobs:
  - build-and-test
  - deploy:
      requires:
        - build-and-test

jobs:
  - build-and-test
  - deploy:
      requires:
        - build-and-test

jobs:
  - build-and-test
  - deploy:
      requires:
        - build-and-test

jobs:
  - build-and-test
  - deploy:
      requires:
        - build-and-test

The

jobs:
  - build-and-test
  - deploy:
      requires:
        - build-and-test

ssh-keygen -t rsa -b 2048

jobs:
  - build-and-test
  - deploy:
      requires:
        - build-and-test

You should now be able to deploy your application to PythonAnywhere using CircleCI.

As described previously, the deploy job is run after the build-and-test job. This is because the deploy job requires the build and test job to pass and will fail if the build-and-test job fails.

Reloading the application on PythonAnywhere

To verify that your application is successfully deployed to PythonAnywhere, you need to manually reload using the PythonAnywhere dashboard. You can avoid this step by creating a bash script inside the PythonAnywhere account. The script automatically reloads the application after a deployment. Call this script

ssh-keygen -t rsa -b 2048

ssh-keygen -t rsa -b 2048

#!/bin/bash
touch var/www/<username>_pythonanywhere_com_wsgi.py

After you add the script to

ssh-keygen -t rsa -b 2048

import sys

# add your project directory to the sys.path
project_home = '/home/waweru/automating-flask-deployments-with-pythonanywhere'
if project_home not in sys.path:
    sys.path = [project_home] + sys.path

# import flask app but need to call it "application" for WSGI to work
from run import app as application  

This command makes the script executable in bash every time changes are detected in PythonAnywhere. Now, every time you start a deploy, the script will be executed and the application will be reloaded. Congratulations on achieving our goal of automating deployments with PythonAnywhere!

Conclusion

Through this tutorial you have been able to learn how to prepare an application for deployment, configure a PythonAnywhere environment, and automate deployment using SSH. This tutorial also explained the importance of SSH keys in deploying to PythonAnywhere. You added to your knowledge of CI/CD practices by setting up your application to deploy if all parameters, including passing tests and CircleCI jobs, are met. And you learned how to do auto-reloads on PythonAnywhere after each deployment. I hope you enjoyed this tutorial can use it with your team to level up your CI/CD practice. Until next time, keep coding!

Waweru Mwaura is a software engineer and a life-long learner who specializes in quality engineering. He is an author at Packt and enjoys reading about engineering, finance, and technology. You can read more about him on his web profile.

How do I turn off Mkvirtualenv?

How do I open virtualenv in PythonAnywhere?

How do I create a virtual environment in PythonAnywhere?

How do I remove a virtualenv from PythonAnywhere?