...
Objectives
...
Managed
...
connector
...
to enable users to experience data sharing with usage control
...
User should be able to sign up and create a connector on the server managed by DG
Usage control could be simple - an application has access to the raw data
Data could be any dummy data
Chain of thoughts during discussion
Selfcurrent process of self-hosted: go to server (SSH) -> run participant software installer → generates generate UI → goes to server → fires up connectors connector → run connectors → data exchange through of CSV files or json
One server (EC2 instance) could be used for multiple connectors hosted for multiple
/singleusersTest the capacity of instance and configuration for a limit on connectors that could be hosted
Experiment on possibilities and limitations of this idea
Take 4 GB of RAM and observe number of connectors
Technologies to be used:
Docker containers with or without Kubernetes
[ OR ]
Kubernetes only
Limit of connectors per user
Have to decide on a number?
Server (EC2 instance) will be destroyed after all connectors running inside are stoppedShould be automated
Multi-threaded creation of connectors
How should it be managed?
Technology?
System Design?
Database can be anywhere
Need more info?
Timeline of each connector
Destroy the connector after it’s its time is over
Expanding capacity using Lambda
Need more info?
Celery worker to observe/ manage visualization
Finalize on class of machines for AWS instance
...