Puzzle ITC - Notice history

All systems operational

Operational

OpenShift 4 Cluster cloudscale.ch - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

OpenShift 4 Cluster LPG1 cloudscale.ch - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021
Operational

Puzzle Website - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

PuzzleTime - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

Puzzle SSO (Keycloak) - Operational

99% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 97.58%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

PuzzleSkills - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

Harbor - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

Grafana - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

Puzzle News Site - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

Zammad - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

CardDAV - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

Argo CD - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

Vault - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

Loki - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

LDAP - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

Puzzle Docs - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

Odoo - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

Netbox - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

PostgreSQL - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021
Operational

Network Office-Bern - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 99.99%Feb · 98.64%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

Network Office-Zürich - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

Google Meet Hardware - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

Telephone Setup - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021
Operational

cloudscale.ch - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

Amazon AWS - Operational

100% - uptime
Dec 2020 · 100.0%Jan 2021 · 100.0%Feb · 100.0%

Previous page

Next page

Dec 2020
Jan 2021
Feb 2021

Notice history

Feb 2021

Jenkins Prometheus Plugin downgrade
  • Completed
    26. February 2021 at 11:6
    Completed
    26. February 2021 at 11:6

    Maintenance has completed successfully.

  • Planned
    26. February 2021 at 11:0
    Planned
    26. February 2021 at 11:0

    We will downgrade the Prometheus Plugin. Jenkins will be restarted during this time.

/sys Wartungsfenster
  • Completed
    25. February 2021 at 20:58
    Completed
    25. February 2021 at 20:58

    All maintenances have been completed successfully and OpenProject has been upgraded to the newest major release. In case you notice any irregularities please contact our helpdesk at help (at) puzzle (dot) ch

  • In progress
    25. February 2021 at 17:8
    In progress
    25. February 2021 at 17:8

    Maintenance is now in progress.

  • Planned
    25. February 2021 at 17:0
    Planned
    25. February 2021 at 17:0

    As usual we will be conducting our monthly maintenance window to apply updates and configuration enhancements. Services may experience a short interruption and/ or periods of degraded performance. Additionally we will update OpenProject to major version 11.

Interruption of Kubernetes Cluster
  • Resolved
    Resolved

    Stable again. Closing incident

  • Monitoring
    Monitoring

    The master node remains stable, we continue to monitor. All applications running on the k8s cluster are up and running

  • Identified
    Identified

    One of the master nodes was not ready anymore. This resulted in short outage of the Kubernetes control plane. This then (for yet unknwon reason) also affected some of the applicactions. I'm currently investigating what exacly happened. I rebooted the vm and it seems stable again.

  • Investigating
    Investigating

    Our Kubernetes Cluster has currently trouble which seems to affect some application. So far we see outages on SSO and Codi, possible more applications.

    investigation started

Networkchange on Cloudscale VMs with multiple Network Interfaces
  • Completed
    23. February 2021 at 18:7
    Completed
    23. February 2021 at 18:7

    Maintenance has completed successfully.

  • Planned
    23. February 2021 at 17:0
    Planned
    23. February 2021 at 17:0

    We are planning for a scheduled maintenance during that time to add some more routes for Servers with multiple network interfaces.

Rancher Management Server unstable
  • Resolved
    Resolved

    Still stable. Closing this incident.

  • Monitoring
    Monitoring

    Rancher Management Server seems stable again. I rebooted all nodes and I'm monitoring the situation now.

  • Investigating
    Investigating

    We see some restarts of Rancher Podss which leads to an unstable Rancher Management Server and API Access to Downstream Cluster.

    Downstream Cluster (Puzzle Kubernetes Production Cluster) is not affected and all applications running on it are working

Jan 2021

Monthly Maintenance Window /sys
  • Completed
    28. January 2021 at 20:34
    Completed
    28. January 2021 at 20:34

    Maintenance has completed successfully.

  • Planned
    28. January 2021 at 17:0
    Planned
    28. January 2021 at 17:0

    Regular monthly maintenance: 28.01.2021 18:00 - 23:00

Cloudscale Staging Network Changes
  • Completed
    28. January 2021 at 20:34
    Completed
    28. January 2021 at 20:34

    Maintenance has completed successfully. Thanks to cloudscale.ch for the support!

  • Planned
    28. January 2021 at 13:0
    Planned
    28. January 2021 at 13:0

    In preparation for a major networking change in the production environment, we will conduct a maintenance window to test all the necessary steps in the staging environment. These changes are necessary to enable the new OCP4 platform on cloudscale.ch. There is no impact on production servers. Announcement for the production change following.

Access to openshift.aws.puzzle.ch (Certificate error)
  • Resolved
    Resolved

    The Incident was caused by a bug of OpenShift version 4.6.9

  • Identified
    Identified

    There is currently a issue with the "cert-utils-operator" to provide the certificate to the target route: "openshift.aws.puzzle.ch" which leads to a invalid certificate. Other applications on the platform should not be affected by this issue.

    We are continuing to work on a fix for this incident.

cloudscale.ch Restricted Internet Connectivity
  • Resolved
    Resolved

    Internet connectivity is stable again. cloudscale.ch will provide a detailed report about the incident. https://cloudscale-status.net/incident/163

  • Identified
    Identified

    cloudscale.ch is experiencing restricted internet connectivity. Access to applications can be limited. For further information please visit: https://cloudscale-status.net/incident/163

Taiga Update
  • Completed
    22. January 2021 at 17:0
    Completed
    22. January 2021 at 17:0

    Maintenance has completed successfully. Taiga is happily running on v5.5.9 now.

  • In progress
    22. January 2021 at 16:30
    In progress
    22. January 2021 at 16:30

    Maintenance is now in progress.

  • Planned
    22. January 2021 at 16:15
    Planned
    22. January 2021 at 16:15

    We are planning for a scheduled maintenance during that time. Taiga will be updated to the latest release.

Dec 2020

/sys Wartungsfenster
  • Completed
    31. December 2020 at 19:7
    Completed
    31. December 2020 at 19:7

    With that we conclude our final maintenance of 2020. All systems are operational. /sys wishes you a Happy New Year.

  • Planned
    31. December 2020 at 8:0
    Planned
    31. December 2020 at 8:0

    Regular monthly maintenance: 31.12.2020 09:00 - 16:00

Dec 2020 to Feb 2021

Next