You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -36,6 +41,7 @@ The codebase can be better visualized as below:
36
41
-[Networking/Security](#networkingsecurity)
37
42
38
43
# Python Flask API
44
+
39
45
## Premise
40
46
41
47
In this section, we will cover a practical solution for setting up a deployable production environment for a simplified application. This environment consists of a API service deployed on Azure Cloud. The goal is to automate the setup of this production environment using "infrastructure as code" principles. Below are the steps to achieve this:
@@ -45,6 +51,7 @@ In this section, we will cover a practical solution for setting up a deployable
45
51
> Note: The solution development was conducted on a MacBook M1. Therefore, the instructions are tailored for use in a macOS environment or a similar development environment.
46
52
47
53
Before we get started ensure you have below tools setup:
54
+
48
55
### Development
49
56
50
57
-*Python v3.11* - For developing the Flask API application.
@@ -69,33 +76,41 @@ All the tools we have used so far are Free to use for personal usage.
69
76
-*Bandit* - A python lib for static code analysis.
70
77
-*Safety* - A python lib for dependency vulnerability analysis.
71
78
-*Checkov* - IAC vulnerability scanning tool, we have used it for scanning our Dockerfile.
72
-
-*trivy* - For scanning vulnerabilities in docker image even before pushing it to the registry.
79
+
-*trivy* - For scanning vulnerabilities in docker image even before pushing it to the registry.
73
80
-*Owasp Zap* - PenTest
74
81
- Github Secret Scanning
75
82
-*Dependabot and Mergify* - Dependabot bumps the dependencies by creating a PR. This helps us keep our dependencies up to date and avoid vulnerabilities. We also use Mergify to streamline the PR merging process, automating it when all the necessary checks and criteria are satisfied.
83
+
76
84
## Running the API Service and DB locally
85
+
77
86
1. Start Docker Desktop
78
87
2. From the root directory of this repository execute below command:
88
+
79
89
```bash
80
90
make start-app-db
81
91
```
92
+
82
93
The above command uses docker compose to run containerized instance of our API and `postgres-13.5` database and then uploads the mock data into the postgres database.
83
94
84
95
3. Test the application by making API requests. For example:
Bootstrap Infrastructure refers to the essential infrastructure resources that are necessary during the initial provisioning phase and ideally remains unchanged or require infrequent modifications.
Contains terraform code for provisioning Azure VNet, VNet associated infrastructure components and Azure Container Apps for deploying a simple containerized application.
283
314
@@ -322,7 +353,9 @@ app_container_config={
322
353
</details>
323
354
324
355
## CI/CD - Github Actions
356
+
325
357
We use Github Actions for automating below operations:
358
+
326
359
- Provisioning our Azure Infrastructure in all the environments with all the quality and security gates
327
360
- Deploying the Python Flask API in all the environments with all the quality and security gates
328
361
- Testing Dependabot and other PR's and automatically merging once all the success criteria are met
@@ -331,18 +364,20 @@ We use Github Actions for automating below operations:
331
364
332
365
We are using `Github-OIDC`forsecuring the connectivity between the Github Actions and Azure Cloud thus reducing the risk of compromising the credentials. Once the service principal is provisioned by the bootstrap infrastructure you must configure themin the github repository under `Settings > Secrets and Variables > Actions > New Repository Secret`
333
366
334
-

367
+

335
368
336
369
Python Application Pipeline:
337
-

370
+

338
371
339
372
Infrastructure Pipeline:
340
-

373
+

341
374
342
375
# Theoretical Case: Secure Database Access
343
376
344
-
### Secure Access Requirements:
377
+
### Secure Access Requirements
378
+
345
379
Secure Access to the Postgres Database deployed on Azure Cloud requires:
380
+
346
381
- End-to-end auditing capabilities for any operation performed.
347
382
- An automated solution for rotating the database credentials every X number of days and workflow capabilities for user management and manual approval/reviews before an action being taken.
348
383
- The solution should provide zero downtime for the application.
@@ -361,42 +396,49 @@ We would like to use below components of the Azure Cloud for implementing the so
361
396
362
397
The solution can be visualized with the help of below request flow diagram:
363
398
364
-

399
+

365
400
366
401
# Assumptions
402
+
367
403
- The API needs to be publicly accessible.
368
404
- The intendant audience has decent understanding of Azure Cloud, Github Actions, Terraform, Docker.
369
405
- They are using Macbook or similar development environment.
370
406
371
407
# ToDo
372
408
373
409
Below is the list of the things we must do to make the implementation production ready.
410
+
374
411
## Documentation
412
+
375
413
- Add Links to the official documentations.
376
414
377
415
## App Code
416
+
378
417
- Migrate from Pip to modern package managers like `Pipenv` or `Poetry`for better dependency management.
379
-
- Implement Unit Test and E2E test appropriately to adhere `Test Pyramid strategy` thus maturing testing strategy.
418
+
- Implement Unit Test and E2E test appropriately to adhere `Test Pyramid strategy` thus maturing testing strategy.
380
419
- Tagging Strategy: Currently we are using job id for tagging docker images which remains unique across the pipeline execution. A preferred approach would be to use `semver`for versioning the images and API.
381
420
382
421
## IAC
422
+
383
423
- Use `Terratest`for Integration test.
384
424
- Implement `Smoke/E2E` testing for IAC once the Infrastructure is provisioned. Execute on a scheduled event to detect any drift from the desired state defined as IAC.
385
425
- Pass the plan artifact from plan to apply. Terraform best practice.
386
426
- Analyze the pros and cons of TF workspace.
387
427
- Analyze the pros and cons of Splitting the AZ Container Apps Environment provisioning through TF and AZ Container App deployment with configuration through Github Actions and YML configs.
388
428
389
429
## CI/CD
430
+
390
431
- Refactor Github Actions Pipeline code to reduce Boilerplate code and practice DRY.
391
432
- Perform Pen Test after Dev Deployment.
392
433
- Certain organization requires manual approval step before `Prod` deployment and creation of a `Change Request`for auditing purposes.
393
434
Thus it should be implemented for the application and infrastructure deployments.
394
435
- Automate `Chaos Testing` using Simian Army for testing Disaster Recovery strategy.
395
436
396
437
## Networking/Security
438
+
397
439
- Restrict Ingress and Egress to the API.
398
440
- Integrate Web Application Firewall.
399
441
- Integrate API Gateway and API Management with the AZ Container Apps. Use appropriate Authentication and Authorization mechanism to protect the API.
400
442
- Block Public access for the `Dev` and `Pre`. Configure VNet to allow access only within Organizations private network, example once the users are connected to the VPN.
401
443
- Setup `tunnel`for using with the CI/CD pipeline thus allowing access to Dev/Pre environment API for executing tests once deployed.
402
-
- Limit the scope of the Service Principals and roles assigned. Create a separate Service Principal for application deployment.
444
+
- Limit the scope of the Service Principals and roles assigned. Create a separate Service Principal for application deployment.
0 commit comments