Skip to content

Commit 32fa8cf

Browse files
committed
docs(README): fix lint errors
1 parent b449ff2 commit 32fa8cf

File tree

5 files changed

+78
-36
lines changed

5 files changed

+78
-36
lines changed

README.md

Lines changed: 78 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ This repository contains solutions to two distinct tasks:
88
The codebase can be better visualized as below:
99

1010
![Visualization of the codebase](./diagram.svg)
11-
11+
1212
# Table of Content
1313

1414
- [Python Postgres Azure Flask Application](#python-postgres-azure-flask-application)
@@ -19,13 +19,18 @@ The codebase can be better visualized as below:
1919
- [Development](#development)
2020
- [DevSecOps Toolings](#devsecops-toolings)
2121
- [Running the API Service and DB locally](#running-the-api-service-and-db-locally)
22+
- [Running the database](#running-the-database)
23+
- [Running the API service](#running-the-api-service)
24+
- [Install prerequisites](#install-prerequisites)
25+
- [Run the application](#run-the-application)
26+
- [Test the application](#test-the-application)
2227
- [Infrastructure as Code](#infrastructure-as-code)
23-
- [Bootstrap Infrastructure - `/infrastructure/bootstrap`:](#bootstrap-infrastructure---infrastructurebootstrap)
28+
- [Bootstrap Infrastructure - `/infrastructure/bootstrap`](#bootstrap-infrastructure---infrastructurebootstrap)
2429
- [**Chicken and Egg Paradox**](#chicken-and-egg-paradox)
25-
- [Application-Infrastructure `/infrastructure/app-infra`:](#application-infrastructure-infrastructureapp-infra)
30+
- [Application-Infrastructure `/infrastructure/app-infra`](#application-infrastructure-infrastructureapp-infra)
2631
- [CI/CD - Github Actions](#cicd---github-actions)
2732
- [Theoretical Case: Secure Database Access](#theoretical-case-secure-database-access)
28-
- [Secure Access Requirements:](#secure-access-requirements)
33+
- [Secure Access Requirements](#secure-access-requirements)
2934
- [Solution](#solution)
3035
- [Assumptions](#assumptions)
3136
- [ToDo](#todo)
@@ -36,6 +41,7 @@ The codebase can be better visualized as below:
3641
- [Networking/Security](#networkingsecurity)
3742

3843
# Python Flask API
44+
3945
## Premise
4046

4147
In this section, we will cover a practical solution for setting up a deployable production environment for a simplified application. This environment consists of a API service deployed on Azure Cloud. The goal is to automate the setup of this production environment using "infrastructure as code" principles. Below are the steps to achieve this:
@@ -45,6 +51,7 @@ In this section, we will cover a practical solution for setting up a deployable
4551
> Note: The solution development was conducted on a MacBook M1. Therefore, the instructions are tailored for use in a macOS environment or a similar development environment.
4652
4753
Before we get started ensure you have below tools setup:
54+
4855
### Development
4956

5057
- *Python v3.11* - For developing the Flask API application.
@@ -69,33 +76,41 @@ All the tools we have used so far are Free to use for personal usage.
6976
- *Bandit* - A python lib for static code analysis.
7077
- *Safety* - A python lib for dependency vulnerability analysis.
7178
- *Checkov* - IAC vulnerability scanning tool, we have used it for scanning our Dockerfile.
72-
- *trivy* - For scanning vulnerabilities in docker image even before pushing it to the registry.
79+
- *trivy* - For scanning vulnerabilities in docker image even before pushing it to the registry.
7380
- *Owasp Zap* - PenTest
7481
- Github Secret Scanning
7582
- *Dependabot and Mergify* - Dependabot bumps the dependencies by creating a PR. This helps us keep our dependencies up to date and avoid vulnerabilities. We also use Mergify to streamline the PR merging process, automating it when all the necessary checks and criteria are satisfied.
83+
7684
## Running the API Service and DB locally
85+
7786
1. Start Docker Desktop
7887
2. From the root directory of this repository execute below command:
88+
7989
```bash
8090
make start-app-db
8191
```
92+
8293
The above command uses docker compose to run containerized instance of our API and `postgres-13.5` database and then uploads the mock data into the postgres database.
8394

8495
3. Test the application by making API requests. For example:
8596
- GET - Greeting API (Health check)
97+
8698
```bash
8799
curl http://127.0.0.1:3000
88100
```
89101

90102
```json
91103
{"message":"Hello world!"}
92104
```
105+
93106
- GET - Rates API
107+
94108
```bash
95109
curl "http://127.0.0.1:3000/rates?date_from=2021-01-01&date_to=2021-01-31&orig_code=CNGGZ&dest_code=EETLL"
96110
```
97111

98112
The output should be something like this:
113+
99114
```json
100115
{
101116
"rates" : [
@@ -113,71 +128,79 @@ All the tools we have used so far are Free to use for personal usage.
113128
]
114129
}
115130
```
131+
116132
4. Stop the application
133+
117134
```bash
118135
make stop-app-db
119136
```
137+
120138
5. Check all available options
139+
121140
```bash
122141
make help
123142
```
143+
124144
</details>
125145
<details>
126146
<summary>Click here to check the local execution steps</summary>
127-
128-
#### Running the database
147+
148+
#### Running the database
129149

130150
There’s an SQL dump in `db/rates.sql` that needs to be loaded into a PostgreSQL 13.5 database.
131-
151+
132152
After installing the database, the data can be imported through:
133-
153+
134154
```bash
135155
createdb rates
136156
psql -h localhost -U postgres < db/rates.sql
137157
```
138-
158+
139159
You can verify that the database is running through:
140-
160+
141161
```bash
142162
psql -h localhost -U postgres -c "SELECT 'alive'"
143163
```
144-
164+
145165
The output should be something like:
146-
166+
147167
```bash
148168
?column?
149169
----------
150170
alive
151171
(1 row)
152172
```
153-
154-
#### Running the API service
155-
173+
174+
#### Running the API service
175+
156176
Start from the `rates` folder.
157-
158-
##### Install prerequisites
159-
177+
178+
##### Install prerequisites
179+
160180
```
161181
DEBIAN_FRONTEND=noninteractive apt-get update && apt-get install -y python3-pip
162182
pip install -U gunicorn
163183
pip3 install -Ur requirements.txt
164184
```
165-
166-
##### Run the application
185+
186+
##### Run the application
187+
167188
```
168189
gunicorn -b :3000 wsgi
169190
```
170-
191+
171192
The API should now be running on [http://localhost:3000](http://localhost:3000).
172-
173-
##### Test the application
174-
193+
194+
##### Test the application
195+
175196
Get average rates between ports:
197+
176198
```
177199
curl "http://127.0.0.1:3000/rates?date_from=2021-01-01&date_to=2021-01-31&orig_code=CNGGZ&dest_code=EETLL"
178200
```
179-
201+
180202
The output should be something like this:
203+
181204
```json
182205
{
183206
"rates" : [
@@ -195,15 +218,16 @@ All the tools we have used so far are Free to use for personal usage.
195218
]
196219
}
197220
```
221+
198222
</details>
199223

200224
## Infrastructure as Code
201-
The solution uses Terraform (infrastructure as code components) that allow you to deploy this environment on cloud providers such as Azure.
202225

226+
The solution uses Terraform (infrastructure as code components) that allow you to deploy this environment on cloud providers such as Azure.
203227

204228
We have 2 logical segregation of the terraform code as below:
205229

206-
### Bootstrap Infrastructure - `/infrastructure/bootstrap`:
230+
### Bootstrap Infrastructure - `/infrastructure/bootstrap`
207231

208232
Bootstrap Infrastructure refers to the essential infrastructure resources that are necessary during the initial provisioning phase and ideally remains unchanged or require infrequent modifications.
209233

@@ -241,25 +265,31 @@ Follow below steps for provisioning bootstrap infrastructure:
241265
```
242266
243267
- From the terminal change the directory
268+
244269
```bash
245270
cd infrastructure/bootstrap
246271
```
272+
247273
- Comment terraform backend config `backend "azurerm" {}` is commented in `infrastructure\bootstrap\main.tf`
248274
249275
- Initialize Terraform for dev environment
276+
250277
```bash
251278
terraform init -var-file=./dev/terraform.tfvars
252279
```
253280
254281
- Plan the Terraform changes and review
282+
255283
```bash
256284
terraform plan -var-file=./dev/terraform.tfvars
257285
```
258286
259287
- Apply changes after its reviewed
288+
260289
```bash
261290
terraform init -var-file=./dev/terraform.tfvars
262291
```
292+
263293
- Re Initialize Terraform to use a remote backend
264294
265295
Uncomment `# backend "azurerm" {}`
@@ -269,6 +299,7 @@ Follow below steps for provisioning bootstrap infrastructure:
269299
```bash
270300
terraform init -backend-config=./dev/backend-config.hcl -var-file=./dev/terraform.tfvars
271301
```
302+
272303
Once successfully executed the local `terraform.state` file has been securely stored in the Azure Storage Account.
273304
274305
- Repeat the steps for other environments.
@@ -277,7 +308,7 @@ We have configured a `bootstrap-infrastructure` Github Actions Pipeline to autom
277308
278309
</details>
279310
280-
### Application-Infrastructure `/infrastructure/app-infra`:
311+
### Application-Infrastructure `/infrastructure/app-infra`
281312
282313
Contains terraform code for provisioning Azure VNet, VNet associated infrastructure components and Azure Container Apps for deploying a simple containerized application.
283314
@@ -322,7 +353,9 @@ app_container_config={
322353
</details>
323354
324355
## CI/CD - Github Actions
356+
325357
We use Github Actions for automating below operations:
358+
326359
- Provisioning our Azure Infrastructure in all the environments with all the quality and security gates
327360
- Deploying the Python Flask API in all the environments with all the quality and security gates
328361
- Testing Dependabot and other PR's and automatically merging once all the success criteria are met
@@ -331,18 +364,20 @@ We use Github Actions for automating below operations:
331364

332365
We are using `Github-OIDC` for securing the connectivity between the Github Actions and Azure Cloud thus reducing the risk of compromising the credentials. Once the service principal is provisioned by the bootstrap infrastructure you must configure them in the github repository under `Settings > Secrets and Variables > Actions > New Repository Secret`
333366

334-
![](./github-secrets.png)
367+
![](docs/images/github-secrets.png)
335368

336369
Python Application Pipeline:
337-
![](/app-pipeline.png)
370+
![](docs/images/app-pipeline.png)
338371

339372
Infrastructure Pipeline:
340-
![](./infra-pipeline.png)
373+
![](docs/images/infra-pipeline.png)
341374

342375
# Theoretical Case: Secure Database Access
343376

344-
### Secure Access Requirements:
377+
### Secure Access Requirements
378+
345379
Secure Access to the Postgres Database deployed on Azure Cloud requires:
380+
346381
- End-to-end auditing capabilities for any operation performed.
347382
- An automated solution for rotating the database credentials every X number of days and workflow capabilities for user management and manual approval/reviews before an action being taken.
348383
- The solution should provide zero downtime for the application.
@@ -361,42 +396,49 @@ We would like to use below components of the Azure Cloud for implementing the so
361396

362397
The solution can be visualized with the help of below request flow diagram:
363398

364-
![](db-secure-access-explained.jpg)
399+
![](docs/images/db-secure-access-explained.jpg)
365400

366401
# Assumptions
402+
367403
- The API needs to be publicly accessible.
368404
- The intendant audience has decent understanding of Azure Cloud, Github Actions, Terraform, Docker.
369405
- They are using Macbook or similar development environment.
370406

371407
# ToDo
372408

373409
Below is the list of the things we must do to make the implementation production ready.
410+
374411
## Documentation
412+
375413
- Add Links to the official documentations.
376414

377415
## App Code
416+
378417
- Migrate from Pip to modern package managers like `Pipenv` or `Poetry` for better dependency management.
379-
- Implement Unit Test and E2E test appropriately to adhere `Test Pyramid strategy` thus maturing testing strategy.
418+
- Implement Unit Test and E2E test appropriately to adhere `Test Pyramid strategy` thus maturing testing strategy.
380419
- Tagging Strategy: Currently we are using job id for tagging docker images which remains unique across the pipeline execution. A preferred approach would be to use `semver` for versioning the images and API.
381420

382421
## IAC
422+
383423
- Use `Terratest` for Integration test.
384424
- Implement `Smoke/E2E` testing for IAC once the Infrastructure is provisioned. Execute on a scheduled event to detect any drift from the desired state defined as IAC.
385425
- Pass the plan artifact from plan to apply. Terraform best practice.
386426
- Analyze the pros and cons of TF workspace.
387427
- Analyze the pros and cons of Splitting the AZ Container Apps Environment provisioning through TF and AZ Container App deployment with configuration through Github Actions and YML configs.
388428

389429
## CI/CD
430+
390431
- Refactor Github Actions Pipeline code to reduce Boilerplate code and practice DRY.
391432
- Perform Pen Test after Dev Deployment.
392433
- Certain organization requires manual approval step before `Prod` deployment and creation of a `Change Request` for auditing purposes.
393434
Thus it should be implemented for the application and infrastructure deployments.
394435
- Automate `Chaos Testing` using Simian Army for testing Disaster Recovery strategy.
395436

396437
## Networking/Security
438+
397439
- Restrict Ingress and Egress to the API.
398440
- Integrate Web Application Firewall.
399441
- Integrate API Gateway and API Management with the AZ Container Apps. Use appropriate Authentication and Authorization mechanism to protect the API.
400442
- Block Public access for the `Dev` and `Pre`. Configure VNet to allow access only within Organizations private network, example once the users are connected to the VPN.
401443
- Setup `tunnel` for using with the CI/CD pipeline thus allowing access to Dev/Pre environment API for executing tests once deployed.
402-
- Limit the scope of the Service Principals and roles assigned. Create a separate Service Principal for application deployment.
444+
- Limit the scope of the Service Principals and roles assigned. Create a separate Service Principal for application deployment.
File renamed without changes.
File renamed without changes.
File renamed without changes.

0 commit comments

Comments
 (0)