Technical Requirements

Goals

Self-hosted Managed Connector that will be used by multiple users for hosting their own connector

Chain of thoughts during discussion

• Self-hosted participant software installer →. generates UI → goes to server → fires up connectors → run connectors → data exchange through CSV files

• One server (EC2 instance) used for multiple connectors hosted for multiple/single users

  • Test the capacity of instance and configuration for a limit on connectors that could be hosted

  • Experiment on possibilities and limitations of this idea

    • Take 4 GB of RAM and observe number of connectors

  • Technologies to be used:

    • Docker containers with or without Kubernetes

    • [ OR ]

    • Kubernetes only

• Limit of connectors per user

  • Have to decide on a number?

• Server (EC2 instance) will be destroyed after all connectors running inside are stopped

  • Should be automated

• Multi-threaded creation of connectors

  • How should it be managed?

  • Technology?

  • System Design?

• Database can be anywhere

  • Need more info?

• Timeline of each connector

  • Destroy the connector after it’s time is over

• Expanding capacity using Lambda

  • Need more info?

• Celery worker to observe/ manage visualization

• Finalize on class of machines for AWS instance

Working Pieces Readily Available

• setup.py that does all of the above (however, have to experiment for scalablity) (link)

• Loom video by sagar (link)

• Video by Waseem (link)

✅ Action Items

🗂 References and documentation

• A Complete Primer on Terraform: link

• A Complete Tutorial on Ansible: link