Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This template is brought to you by Lucidchart, a diagramming app for Confluence.

...

Diagram owner

...

Team

...

Informed

...

Status

...

Status
colourBlue
titledraft
/
Status
colourYellow
titlein review
/
Status
colourGreen
titleapproved
/
Status
colourPurple
titleIn production

...

Last date updated

...

...

On this page

...

Table of Contents
minLevel2
maxLevel2

...

titlePillars of the AWS Well-Architected Framework

...

Name

...

Description

...

Operational Excellence

...

The ability to run and monitor systems to deliver business value and to continually improve supporting processes and procedures.

...

Security

...

The ability to protect information, systems, and assets while delivering business value through risk assessments and mitigation strategies.

...

Reliability

...

The ability of a system to recover from infrastructure or service disruptions, dynamically acquire computing resources to meet demand, and mitigate disruptions such as misconfigurations or transient network issues.

...

Performance Efficiency

...

The ability to use computing resources efficiently to meet system requirements, and to maintain that efficiency as demand changes and technologies evolve

...

Cost Optimization

...

The ability to run systems to deliver business value at the lowest price point.

AWS Well Architected Framework PDF

Goals

📐 Architecture

<Add diagram>

Install your favorite diagramming app from our Marketplace, then use the / command to embed an architecture diagram in this table. Try using the Lucidchart AWS diagram.

Architecture flow

Other notes:

🚀 Deployment strategy

Deployment region:

⌨ Rest API details

Code Block
{
  "samplecode: string"
}

🤝 SLA

Requests Per Second (RPS):

...

Objectives

  • Managed connector to enable users to experience data sharing with usage control

    • User should be able to sign up and create a connector on the server managed by DG

    • Usage control could be simple - an application has access to the raw data

    • Data could be any dummy data

Chain of thoughts during discussion

  • current process of self-hosted: go to server (SSH) -> run participant software installer → generate UI → fires up connector → run connectors → data exchange of CSV files or json

  • One server (EC2 instance) could be used for multiple connectors hosted for multiple/single users

    • Test the capacity of instance and configuration for a limit on connectors that could be hosted

    • Experiment on possibilities and limitations of this idea

      • Take 4 GB of RAM and observe number of connectors

    • Technologies to be used:

      • Docker containers with or without Kubernetes

      • [ OR ]

      • Kubernetes only

  • Limit of connectors per user

    • Have to decide on a number?

  • Server (EC2 instance) will be destroyed after all connectors running inside are stopped

    • Should be automated

  • Multi-threaded creation of connectors

    • How should it be managed?

    • Technology?

    • System Design?

  • Timeline of each connector

    • Destroy the connector after its time is over

  • Expanding capacity using Lambda

    • Need more info?

  • Celery worker to observe/ manage visualization

  • Finalize on class of machines for AWS instance

Working Pieces Readily Available

  • setup.py that does all of the above (however, have to experiment for scalablity) (link)

  • Loom video by sagar (link)

  • Video by Waseem (link)

✅ Action Items

Action

Description

Owner

Due date

Jira ticket

1
  •  Test server for capacity of connectors it can run

2
  •  Experiment with Docker containers to run multiple containers

3
  •  Check Kubernetes can solve the problem of managing multiple connectors on a server/ multiple servers

🗂 References and documentation

  • A Complete Primer on Terraform: link

  • A Complete Tutorial on Ansible: link