Tutorial on Using Gitlab CI/CD Pipelines to Deploy Your Python Flask Restful API With Postgres on Heroku
Today we will build a Restful API using Python Flask, SQLAlchemy using Postgres as our Database, testing using Python Unittest, a CI/CD Pipeline on Gitlab, and Deployment to Heroku.
If you don’t have an account already, Heroku offer’s 5 free applications in their free tier account. Once you have created your account, create 2 applications. I named mine flask-api-staging and flask-api-prod.
You can create the applications via cli or the ui, from the ui it will look more or less like this:
Select an app name and check if the name is available then select create. Note down the name and config as we will use it in our .gitlab-ci.yml config:
Heroku API Key
To allow the deployment of applications to Heroku from Gitlab, we need to generate a API Key on Heroku and save the config in Gitlab.
Head over to your Heroku Dashboard, select Account Settings, scroll to the API Key section and generate a API Key.
Head over to your Gitlab Repository, select Settings, CI/CD, then select Variables enter the Key: HEROKU_API_KEY and the Secret of the API Key into the Value and select Save Variable.
We will reference this variable from our deploy steps.
Heroku Postgres Add-on
Heroku offers a free Postgres Add-On, to activate: Select your application, select Resources, search for the Add-on Heroku Postgres, select and select the Hobby Dev Free version and select provision.
Our Application Code
Clone your repository then let’s start by creating our Flask API. Note this is more on Gitlab CI/CD than going into detail into the Flask Application.
Note that we are using flask-heroku, with this package Heroku will automatically discover your configuration for your database using environment variables. So if you have a postgres add-on, you don’t need to specify the location of your database.
If you want to use sqlite, you can remove the heroku instantiation and uncomment the SQLALCHEMY_DATABASE_URI property in your config.cfg
fromflaskimportFlask,jsonify,requestfromflask_sqlalchemyimportSQLAlchemyfromflask_marshmallowimportMarshmallowfromflask_herokuimportHerokufrompasslib.hashimportsha256_cryptfromdatetimeimportdatetimeapp=Flask(__name__)app.config.from_pyfile('config.cfg')heroku=Heroku(app)db=SQLAlchemy(app)ma=Marshmallow(app)## --Database Models--classMember(db.Model):__tablename__='members'id=db.Column(db.Integer,primary_key=True,autoincrement=True)email=db.Column(db.String(255),unique=True,nullable=False)username=db.Column(db.String(50),unique=True)password_hash=db.Column(db.String(100))firstname=db.Column(db.String(50),unique=False)lastname=db.Column(db.String(50),unique=False)registered_on=db.Column(db.DateTime,nullable=False)classMemberSchema(ma.ModelSchema):classMeta:model=Memberfields=('id','username','email')member_schema=MemberSchema(strict=True,only=('id','username'))members_schema=MemberSchema(strict=True,many=True)## --Views--@app.route('/')defindex():returnjsonify({'message':'ok'}),200# list users@app.route('/api/user',methods=['GET'])deflist_users():all_users=Member.query.all()result=members_schema.dump(all_users)returnjsonify(result.data)# get user@app.route('/api/user/<int:id>',methods=['GET'])defget_user(id):user=Member.query.get(id)result=member_schema.dump(user)returnjsonify(result.data)# add user@app.route('/api/user',methods=['POST'])defadd_user():email=request.json['email']username=request.json['username']password_hash=sha256_crypt.encrypt(request.json['password'])firstname=request.json['firstname']lastname=request.json['lastname']new_user=Member(email=email,username=username,password_hash=password_hash,firstname=firstname,lastname=lastname,registered_on=datetime.utcnow())try:db.session.add(new_user)db.session.commit()result=member_schema.dump(Member.query.get(new_user.id))returnjsonify({'member':result.data})except:db.session.rollback()result={'message':'error'}returnjsonify(result)# update user@app.route('/api/user/<int:id>',methods=['PUT'])defupdate_user(id):user=Member.query.get(id)username=request.json['username']email=request.json['email']user.email=emailuser.username=usernamedb.session.commit()returnmember_schema.jsonify(user)# delete user@app.route('/api/user/<int:id>',methods=['DELETE'])defdelete_user(id):user=Member.query.get(id)db.session.delete(user)db.session.commit()returnjsonify({'message':'{} has been deleted'.format(user.username)})if__name__=='__main__':app.run()
And lastly, our gitlab-ci configuration which will include our build, test and deploy steps. As soon as a commit to master is received the pipeline will be acticated. Note that our production deploy step is a manual trigger.
Our config for .gitlab-ci.yml. Note to replace your Heroku app names.
Once the code has been pushed to master, gitlab will pick it up and trigger the pipeline to run.
Gitlab Pipelines
Head over to Gitlab, select CI/CD -> Pipelines, you should see a running pipeline, select it, then you should see the overview of all your jobs:
If everything has passed you should see the Passed status as shown above.
You will notice that the staging environment has been deployed. Now you can do some testing and when you are happy with it, you can select the play button which will deploy to production on the pipelines dashboard.
Creating the Tables on Postgres
Before we can interact with our API, we need to provision the postgres tables from the database models that we wrote in our application.
Open up a Python shell on Heroku and initialize the tables:
1234
$ heroku run python -a flask-api-prod
>>> from app import db
>>> db.create_all()>>> exit()
Testing the API:
Now that everything is up and running, its time to test our API.
Please feel free to show support by, sharing this post, making a donation, subscribing or reach out to me if you want me to demo and write up on any specific tech topic.