Farmstack Tech Master Doc

Modules:

1. FS Core

2. FS Central

3. FS Participant

Points to work on:

1. Architecture of FS and tools required

   1. Service oriented architecture vs Monolithic

   2. Entities in FS - attributes, functions, operations

2. Database structure of FS entities (about connectors, policies, logs, users etc)

3. Wrappers around the entities available from Fraunhofer

Architecture of FS (other than connectors):

1. Decision points:

   1. Service oriented vs monolith vs something else

      1. Is distributed DB must?

      2. Trade off: testability/ deployability vs faster development time

      3. Feasibility: Distributed system

      4. Collaboration: with dev partners like Fraunhofer and other potential partners

      5. Possibility to refactor entire codebase later

2. Tools

   1. Deployment scripts/commands/plugins

   2. Data provider and consumer custom applications (ex: NodeJS, Python, Java).

   3. User management frontend and backend.

   4. User console for:

      1. Deploying instances

      2. Interacting with deployment servers.

      3. Log for the user for services.

Important entities of FS:

1. Connectors:

   1. Fraunhofer maintains the information model (declarative as well as programmatic) of the connectors/ usage policies/ contracts etc which defines the fields and attributes

   2. DG should maintain:

      1. Functionality to extract the attributes and create entries in DB that can be referenced later

      2. Configurable templates of the connectors

      3. Wrapper to easily instantiate connector(s) from the templates

      4. Functionality to orchestrate through a frontend UI

   3. Important categories of attributes:

      1. Type (provider vs consumer)

      2. Status (active, inactive, paused)

      3. Certificate

      4. Owner (org, person, certificate etc)

      5. Structure (container #s, sequence etc)

      6. Usage policies (type, metadata, status etc)

1. Certificates (in the section below CA refers to certificate authority and CnA will be Certification Authority):

   1. Open questions:

      1. Certification Vs Certificate Authority - what Fraunhofer does maintain is certification authority (authority on process of creating certificates rather than creating certificates)

         1. We don’t need certification from IDSA to operate a connector

      2. Will DG + Fraunhofer be a CA or CnA?

         1. No, DG won’t be CA. The participant organisation doesn’t need a certificate since the access is given after manual verification

      3. What are the services to obtain certificates for being a CA? Or is it a manual process?

         1. Question is redundant as DG is not becoming a CA. FS needs to provide way to install TLS certificates to the connectors which can be obtained from ACME2 and a CA that needs to be decided (paid service)

      4. Do the certificates need to be part of DB maintained by FS central/ core?

         1. No, don’t think FS needs to have DB of certificates

         2. ??FS might need a DB of metadata of certificates??

2. Policies:

   1. Fraunhofer maintains the information model that is serialized in json-ld

   2. DG maintains:

      1. Define the workflow -> web sequence diagram for FS

      2. Functionality to extract the attributes and write to DB that can be referenced later

      3. Configurable templates of the policies

      4. ??Wrapper to easily instantiate static policies ??

      5. ??Wrapper for dynamic policies - with Fraunhofer??

      6. Functionality to orchestrate through a frontend UI

   3. Important Attributes:

      1. Type

      2. Status

      3. Certificate

      4. Owner

      5. Structure (can be multiple contracts)

      6. Association with connector

3. Users

4. Organisations

5. Data resource (metadata of data)

6. Package (node/python app + docker container)

   1. Is there a need to prove ownership of a package by the respective connector/ org?

   2. What about the scenarios where the users insist on co-owning (multiple signatures)?

7. Participant host server

8. Central host server

Tools/ services:

1. Logging service:

   1. Functionality

1. Participant logs:

   1. Low level logs (to be flushed out):

      1. Connector logs: Lowest level logs generated to log files through the connector (debug logs)

      2. Package logs: Logs generated by the app while providing/ consuming data to the connector (if any running on the provider)

   2. Participant web console logs: High level logs to be provided to the admin/root of the web admin console (status, data volume,

User Management:

User journey:

1. User signup and onboarding. (Backend for central storage of Users)

2. Pre Provisioning the Deployment Server - Get required server information (SSH key, Cloud Service Provider API Key (Currently for AWS)) from Participant. (We might need some server management tool installed in case of personal/datacenter server)

3. Provision the deployment machines using some kind of middleware (AWS CLI, Terraform or other third party interfacing tool), install prerequisites and configure settings for deployment.

4. Containerize the application (R&D required)

5. Generate certificates, configurations - policies, uc, authentication

6. Configure required yml and xml files. (including data sources)

7. Clone the fs-core (connectors) on the provisioned server(s).

8. Attach the generated configurations to the cloned fs-core.

9. Run the docker machine.

Section 1 - User Signup and Onboarding:

Technologies under evaluations:

Evaluation Requirements:

Results:

Final Selection:

Section 3 - Provisioning the deployment servers:

Technologies under evaluations:

Evaluation Requirements:

Results:

Final Selection:

Section 4 - Containerize the application:

Technologies under evaluations:

Evaluation Requirements:

Results:

Final Selection:

Section 5 - Generation of Configuration

Technologies under evaluations:

Evaluation Requirements:

Results:

Final Selection:

Section 6 - Configure required yml and xml files

Technologies under evaluations:

Evaluation Requirements:

Results:

Final Selection:

Section 7 - Clone fs-core (connectors) image on the provisioned server

Technologies under evaluations:

Evaluation Requirements:

Results:

Final Selection:

Section 8 - Attach the generated configurations to the cloned fs-core

Technologies under evaluations:

Evaluation Requirements:

Results:

Final Selection:

Section 9 - Run the docker machine

Technologies under evaluations:

Evaluation Requirements:

Results:

Final Selection: