2019-09-16 GOBii-PFR Hackathon, system overview, structure; GOBii locally w/ Docker;Bitbucket builds; Kube deploymnt;Deplmnt GOBii loader web; Keycloak trials.

Sep 16, 2019

Overview of the system and current structure

• A+A system(s)?

  • Believe in the last version i saw this was built in. Consider external A+A like Redhats Keycloak.

  • We have authorization built-in (roles, permissions – albeit not fully implemented) but dependent on external authentication (LDAP) and has proved to be problematic. Adapting something like Keycloak would be a great addition.

• Image repo - dockerhub?

  • Yes, already in dockerhub.

• Meet the team, overview of the week aheads goal, standup

• @KevinPalis Demonstrated the GDM Deployment and associated systems.

  • Configuration/data baked in to images.

  • Future Plan of utilising more cloud services (Lambda etc, however requirement to keep on-prem an cloud infrastructure similar makes this difficult)

  • Shared volume mount between nodes due to size requirements.

• @Lee O'Grady Demonstrated PFRs KiwiCloud Concept and the infrastructure behind it.

  • Justification of Kubernetes in this platform due to number of moving parts

  • Easier for PFR as full control over underlying infrastructure. GOBii has to meet different clients needs.

• Utilise remaining hour to do some code

Sep 17, 2019

Run GOBii locally - full stack up with docker-compose for development.

• Yes, I’ve been looking into docker-compose myself and want to try if it can handle our use-cases.

• Discussion on Keycloak and its capabilities.

• Debugging of connections on AWS deployment

  • Docker Networks implementation to solve inefficient networking, configuration & firewall issues

  • Rolling AWS EC2 (new IPs) now not a issue

• Overview of the Ansible setup (work in progress)

• Review gobiidockers repo + Bamboo

  • From this review its clear that pipelines cannot deal with the complexity involved in bamboo.

  • Discussion spin off on using a 4th container that supplies the remote volume configuration steps and seed data requirements

  • Discussion around architecture of current system and its fit with compose/Ansible/dockerfiles.

• docker-compose setup

  • Requires autostart sldap

  • Add docker restart policy to deployment scripts

  • Load container config during `compose up`

  • Challenges encountered here while looking at the configuration steps.

Sep 18, 2019

Bitbucket builds & publish image

• Not Bitbucket but Bamboo. However, all source code are in Bitbucket.

• PFR Access to Bamboo for troubleshooting builds (discuss Pipelines as alternate)

• Due to the build process in bamboo pipelines are not feasible at this stage.

• Access to Bamboo is not required.

• Loader UI under development by PFR will utilise Bitbucket Pipelines due to having a simpler build process.

Sep 18, 2019

Deploy to Kube with Helm (on AWS as PFR have that available, but this is technically irrelevant).

• Provide some of the cluster setup scripting - This is probably going to be later. Shamim is working on Terraforming our cluster.

• Sounds good. Although I am also looking into Docker swarm as it looks very promising as of the latest release.

• Unrealistic idea, this will require some thought around some of the ground choices.

Sep 18, 2019

Source code, image/package repos & deployments of Gobii Loader Web

Source code: Bitbucket

Keycloak Theme: Bitbucket

Package Repo: NPM (PFRs)

Image Repo: Dockerhub

PFR QA Deployment: AWS

Sep 18, 2019

CI/CD Pipeline with Jenkins?

• Already in Bamboo. No need for Jenkins.

CI pipeline in Bitbucket Pipelines for gobii-loader-ui provided by PFR.
Documentation in the repo explains how to deploy, alternatively the docker-compose setup allows the stack including the loader to be initialised.

Lee demonstrated the CI system to be used for the Angular deployments.

Sep 18, 2019

Review the mockup for Portal and Job Status

Constructive session, lots of thoughts shared, some future features discussed and scheduled to be added to the backlog.

Sep 18, 2019

Keycloak Trials

• Connection to LDAP & test password management

• POC usage from python

@Yaw Nti-Addae proved connection and auth via a python script to local keycloak

@Roy Petrie experimenting with connection to LDAP for password reset etc.

Sep 19, 2019

Configure Keycloak in docker-compose setup

• Definition of the service

• Default user

• Install the Theme

Copies in from local file system, assumes repo for keycloak also checked out, if not no errors.

Manual steps required to configure the usage of the theme and complete setup. will look at a restore of config from file when we know whats needed to be configured.

Sep 19, 2019

Finish Docker Compose setup

• Configure web

Lots of work configuring and simplifying the compose setup

Sep 19, 2019

Nginx Reverse Proxy with Denis

Conference call

Handling paths to ports. Includes SSL Termination. Containerisation is next step.

This solution will enable path based routing. AWS Load balancing is subdomain based.

Sep 19, 2019

Review

@KevinPalis

• Networks to simplify the networking and pull communication internally (immune restart re-config)

• Compose

@Roy Petrie

• Keycloak summary

@Lee O'Grady

• Demo the Angular applications deployed via compose