Panasonic

Enhanced upon the existing CFT, written for dev environments for production. Since, all the environments i.e. 'dev', 'stage', 'qa' and 'prod' referred to the 'master' branch Cloudformation's 'Mapping' and 'Condition' statements were used for bifurcating the configuration of each environments such as minimum and maximum number of tasks, tasks's memory and CPU allocation, database instance sizes etc.

To optimize both cost and performance, auto-scaling capabilities were implemented for Amazon ECS tasks. By setting up auto-scaling policies, the system could dynamically adjust the number of running tasks based on real-time demand. This approach ensured that resources were utilized efficiently, leading to cost savings during low-traffic periods and maintaining performance during peak times.

The AWS ECS, by its nature, distributes the tasks across multiple availability zones. Using this, the tasks were distributed across multiple availability zones to achieve high availability.

The existing GitLab CI/CD pipelines were updated to streamline the deployment processes for both application code and infrastructure changes for the 'production' environment. This involved integrating the CloudFormation templates and auto-scaling configurations into the pipelines.

A major enhancement involved identifying repeating operations such as building the

docker image, pushing it to ECR, deploying to ECS etc. and modularizing those operations in order to make clutter free CI/CD pipelines.. The purpose of doing so was to

make other domains of the project able to utilize those modules, improving overall efficiency.

The improved pipelines enabled automated, consistent deployments, reducing the time and effort required for manual interventions and minimizing deployment errors.

The project faced a significant challenge in managing the deployment of AWS API Gateway due to the large swagger file, the inherent complexities of the AWS API Gateway service, and the need to integrate seamlessly into an existing deployment workflow.

To address these custom deployment requirements, the deployment workflow was updated to include a Lambda function at the deployment stage. This function's primary role was to detect updates to the API document. Upon detecting an update, it would modify the document with the necessary parameters, upload it to S3, and proceed with deploying the API on AWS API Gateway. This approach not only streamlined the deployment process but also ensured that the API Gateway configurations were always up-to-date, minimizing manual intervention and potential errors.

This approach also ensured that no unnecessary deployments were created for API Gateways when the swagger file itself was not updated prior to deployment.

Implemented AWS Secrets Manager for storing the sensitive data such as database password, API Keys and other information in an encrypted manner. These secrets were mounted on each task as an environment variable. Since the Pimcore is able to read the environment variables and utilize it, there was no need of updating the variables with actual values in the source code.

In addition to this, periodic password rotation for the RDS Aurora through a custom AWS Lambda function was implemented.

The RBAC for each cloud entity were carefully audited periodically and necessary changes were made if required.

The Pimcore application relies heavily on the host file system for storing caching and configuration data and in order to achieve certain functionalities. From the containerization perspective, this is an anti-pattern since each container will have it's own identity which would end up in a non uniform user experience. Hence it was required to make the containers immutable.

Doing so required the application refactoring, taking the configuration storage into the database where it is uniformly available for every ECS task.

Also, redis cache was implemented for storing the cache information. Redis also resulted in reducing the workload running on ECS tasks, making them even more performant.

Since the ECS tasks and containers are ephemeral, it was required to export the logs from where it can be retrieved and analyzed. Initially a FluentBit sidecar container which runs along with the application container in each task and keeps pushing the logs to AWS Cloudwatch Logs.

Since, this approach required to have a volume shared with each container i.e. application and logging, it required to add an AWS EFS Volume and mount in both the containers which comes with the manageability overhead, a better approach was required.

The application code was updated to use the logging libraries which have direct support of AWS Cloudwatch logs.