AIC Challenge

 AIC servers were collocated  on 3^rd party data center. The servers included the mix of Application and Database servers to maintain the data, which comprises of applications and the geo-spatial based images, the dynamic data in order to maintain the compete track of their client agriculture insurance and the factors which are essential to maintain the status of the client’s ever changing data, images etc. and due to the limited storage facility available in their existing servers, they were unable to address to meet the growing storage requirements for their data in excess of 100 TB Data. Besides that, AIC also had to address the other factors like operational excellence, reliability, performance, sustainability & cost optimization not only to maintain their existing data needs, but also plan the action point to meet their growing data needs.

 Why AWS Architecture

Use Case : Primary Storage with implementation of AWS FSx ONTAP

 To meet the challenges faced by AIC due to their limited collocated resources, Unicloud Labs proposed the AWS services, based on 6 pillars of Architecture, and migrated the complete Collocated application servers and storage data to AWS cloud. To meet the business requirements, AIC team is using the windows-based  GIS applications servers  such as ArcGIS, ERDAS Imaging, Snap Server by ESA and the GIS based data is processed by these applications and is then output files are stored in storage for referring the output files as and when required. Unicloud proposed to the client that remote sensing applications shall be maintained on the windows operating server and for the database services AWS RDS – MySQL services was recommended. In addition to that for the web specific application servers, Ubuntu OS based EC2 instances were recommended. The AIC repository is a cloud-native solution, with the front-end web app separated from the backend API. It runs as immutable infrastructure in a fully automated environment, with all infrastructure defined in code to support for building and managing online surveys and databases, managing the GIS based data storage & retrieval while using the AWS S3 service.

 To meet the AIC challenges the following AWS services were used.

• Amazon Elastic Compute Cloud (Amazon EC2), which provides scalable computing capacity in the Amazon Web Services (AWS) Cloud. Using Amazon EC2 eliminates your need to invest in hardware up front, so you can develop and deploy applications faster. You can use Amazon EC2 to launch as many or as few virtual servers as you need, configure security and networking, and manage storage. AIC uses this service to host its Microsoft windows-based application servers. This enables AIC to scale up under load and scale down during periods of inactivity to save money and pay for only what it uses.
• Amazon Elastic Load Balancing(ELB), which is used to distribute incoming traffic across multiple Amazon  EC2 instances, as required to achieve redundancy and fault-tolerance. AIC uses this service for load balancing  activities.
• AWS Simple Storage Service (S3) offers a range of storage classes for the objects that you store. You choose a class depending on your use case scenario and performance access requirements. All these storage classes offer high durability. AIC uses this service to store its dynamic & critical data pertaining to the GIS data & market research case studies as primary storge use case. The data backup generated with AWS FSx ONTAP solution  is getting stored in S3 which may be used for data restoration in case of any fatal failure. In addition to that S3 has been configured for the data backup & restoration process by implementing the AWS best recommended practices such as IAM roles & least privilege deployment, bucket versioning, Encryption, S3 Optimization – Tagging, Metrics and Performance Monitoring, Multivolume Backups, Database Replica Backups with proper versioning and defined retention period.  FSx ONTAP has been used for the central data share for the end users, which is mapped with the EC2 and the complete backup of the EC2 and FSx has been configured on S3 path. In case of fatal failure, EC2 and FSx backup  can be restored from the AWS backup saved in S3.
• AWS Managed Microsoft AD which is an  AWS Directory Service for Microsoft Active Directory (Standard Edition or Enterprise Edition) and configured as  actual Microsoft Active Directory in the AWS Cloud. You can use it to support Active Directory workloads; Amazon Relational Database Service for Microsoft SQL Server; AWS managed services such as Amazon Workspaces and Amazon QuickSight; or Linux applications that require an LDAP directory. AIC uses this service  for user authentication using the Windows services.
• Amazon Virtual Private Cloud (Amazon VPC) enables you to launch AWS resources into a virtual network that you've defined. This virtual network closely resembles a traditional network that you'd operate in your own data center, with the benefits of using the scalable infrastructure of AWS. AIC uses this service for the use of dedicated virtual network in its AWS account.
• Amazon Relational Database Service (RDS), which is a web service that makes it easier to set up, operate, and scale a relational database in the AWS Cloud. It provides AIC group the cost-efficient, resizable capacity for an industry-standard relational database and manages common database administration tasks.
• Amazon FSx for NetApp ONTAP, which offers high-performance file storage that is broadly accessible from Linux, Windows, and macOS compute instances via the industry standard NFS, SMB, and iSCSI protocols. It enables you to use ONTAP’s widely adopted data management capabilities, like snapshots, clones, and replication, with the click of a button. In addition, it provides low-cost storage capacity that’s fully elastic and virtually unlimited in size and supports compression and deduplication to help you further reduce storage costs. AIC uses this service for the file share services for the Application Servers among the end users.
• AWS CloudFormation, which enables automated creation, updating, and destruction of AWS resources. AIC group also used the Troposphere library, which enables the creation of objects via AWS CloudFormation using Python instead of hand-coded JSON—each object representing one AWS resource such as an instance, an Elastic IP (EIP) address, or a security group.
• Amazon Cloud Trail (Cloud Trail) is an AWS service that helps you enable governance, compliance, and operational and risk auditing of your AWS account. Actions taken by a user, role, or an AWS service are recorded as events in CloudTrail. AIC used this service for its governance, compliance, and operational activities.

• AWS CloudWatch, which provides a monitoring service for AWS cloud resources and the applications running on AWS. AIC uses this service to monitor the health of the AWS resources along with the optimization.

The Benefits

A Windows based  GIS Application Server(s) with its applications  & dynamic data for the future

 Through its use of AWS, with support of the AWS Services Partner, Unicloud Labs Pvt Ltd, AIC is making its vast wealth of GIS data repository,  for pictures, videos, audio files & case studies —easily discoverable in one centralized location, delivering these benefits:

Easy Access to the products responses. The GIS Application Servers and repository  automatically optimizes the user experience for each user’s case.

  • Built-in Scalability. All components of the AIC  AWS cloud are built to scale on demand, as needed to handle usage spikes. “On-demand scalability will be invaluable for events such as the need for mosaicking  activity, processing of raw GIS data in optimized output files. AIC  has really benefitted not only on operational but also on the productivity front just because of the elastic scalability and exceptional computation which was not possible with limited collocated resources.

 • Good Use of company’s capital. By moving the high-end GIS data-based Application resources, databases in the cloud, AIC avoided the costs associated with deploying and maintaining servers and storage hardware in-collocated environment. Instead, the agency can simply pay for the AWS resources it uses at any given time with an ease without giving a single thought on the high-end capex for resources installations.

While AIC's AWS cloud environment delivers a wealth of new convenience and capabilities, for their end users and keeps on expanding with exceptional results."