Technical Requirements

Goals

Farmstack Managed Connector that will be used by multiple users for hosting their own connector on DG server acrchitecture

Managed connector by Digital Green to enable users to experience data sharing with usage control restricting usage of data by a simple application

Chain of thoughts during discussion

• Self-hosted participant software installer → generates UI → goes to server → fires up connectors → run connectors → data exchange through CSV files

• One server (EC2 instance) used for multiple connectors hosted for multiple/single users

  • Test the capacity of instance and configuration for a limit on connectors that could be hosted

  • Experiment on possibilities and limitations of this idea

    • Take 4 GB of RAM and observe number of connectors

  • Technologies to be used:

    • Docker containers with or without Kubernetes

    • [ OR ]

    • Kubernetes only

• Limit of connectors per user

  • Have to decide on a number?

• Server (EC2 instance) will be destroyed after all connectors running inside are stopped

  • Should be automated

• Multi-threaded creation of connectors

  • How should it be managed?

  • Technology?

  • System Design?

• Database can be anywhere

  • Need more info?

• Timeline of each connector

  • Destroy the connector after it’s time is over

• Expanding capacity using Lambda

  • Need more info?

• Celery worker to observe/ manage visualization

• Finalize on class of machines for AWS instance

Working Pieces Readily Available

• setup.py that does all of the above (however, have to experiment for scalablity) (link)

• Loom video by sagar (link)

• Video by Waseem (link)

✅ Action Items

🗂 References and documentation

• A Complete Primer on Terraform: link

• A Complete Tutorial on Ansible: link