Puzzle ITC - Notice history

All systems operational

Operational

OpenShift 4 Cluster cloudscale.ch - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

OpenShift 4 Cluster LPG1 cloudscale.ch - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 99.78%

Previous page

Next page

Apr 2024
May 2024
Jun 2024
Operational

Puzzle Website (puzzle.ch) - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Mail - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Rocket.Chat - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Taiga - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

PuzzleTime - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Wiki - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Puzzle SSO (Keycloak) - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

CodiMD - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

OpenProject - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Nextcloud - Operational

100% - uptime
Apr 2024 · 99.98%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

PuzzleSkills - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

GitLab - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Jenkins - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Quay (registry.puzzle.ch ) - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Harbor (harbor.puzzle.ch) - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Grafana - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Puzzle News Site (news.puzzle.ch) - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Cryptopus - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Artifactory - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

SonarQube - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Puzzle Meet - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Zammad (help.puzzle.ch) - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Sentry - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

CardDAV - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Webmail - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Argo CD - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Youtrack - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

OKR - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Vault - Operational

99% - uptime
Apr 2024 · 100.0%May · 99.39%Jun · 99.07%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Loki - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

LDAP - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Puzzle Docs (docs.puzzle.ch) - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024
Operational

Network Office-Bern - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Google Meet Hardware - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Telephone Setup - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024
Operational

cloudscale.ch - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Amazon AWS - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Notice history

Jun 2024

/sys Wartungsfenster
  • Completed
    27. June 2024 at 21:18
    Completed
    27. June 2024 at 21:18
    Maintenance has completed successfully.
  • In progress
    27. June 2024 at 16:0
    In progress
    27. June 2024 at 16:0
    Maintenance is now in progress
  • Planned
    27. June 2024 at 16:0
    Planned
    27. June 2024 at 16:0

    As usual we will be conducting our monthly maintenance window to apply updates and configuration enhancements. Services may experience a short interruption and/ or periods of degraded performance.

Migration MX Servers
  • Completed
    27. June 2024 at 16:0
    Completed
    27. June 2024 at 16:0
    Maintenance has completed successfully
  • In progress
    27. June 2024 at 15:0
    In progress
    27. June 2024 at 15:0
    Maintenance is now in progress
  • Planned
    27. June 2024 at 15:0
    Planned
    27. June 2024 at 15:0

    We will be migrating our inbound mail infrastructure (mx servers) today, starting at 5pm. During this time, there might be slight delays of a couple of minutes for inbound messages. Thank you for your understanding.

OpenShift 4 Monthly Maintenance
  • Completed
    24. June 2024 at 16:18
    Completed
    24. June 2024 at 16:18

    The upgrade to 4.15.17 has completed successfully. All services are operational.

  • In progress
    24. June 2024 at 15:0
    In progress
    24. June 2024 at 15:0
    Maintenance is now in progress
  • Planned
    24. June 2024 at 15:0
    Planned
    24. June 2024 at 15:0

    As usual we will be conducting our monthly maintenance window to apply updates and configuration enhancements. Services may experience a short interruption and/or periods of degraded performance.

OpenShift LPG/Staging Cluster Network Outage
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring
    We implemented a fix and are currently monitoring the result.
  • Update
    Update

    Still unclear, what the root cause is. We are analyzing possible workarounds.

  • Investigating
    Investigating

    We are currently investigating this incident. During the cluster upgrade of the LPG/Staging cluster, the network pods are crashlooping.

Outage of files.puzzle.ch
  • Resolved
    Resolved

    We are happy with the results of the mitigation and observed a stable operation over the last hours.

  • Update
    Update

    We took corrective actions and we are monitoring the results. files.puzzle.ch is available again.

  • Monitoring
    Monitoring

    We took corrective actions and we are monitoring the results. files.puzzle.ch is available again.

May 2024

Partial outage of Vault
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring

    Vault was successfully restored to a known good db state of Thursday 30.05.2024 20:11.

    Secrets that were added between 20:11 and the next day around 08:30 (after which the instance was offline) are unfortunately lost.

    We apologize for the inconvenience.

    We are continuing to monitor the performance of the restored cluster.

  • Identified
    Identified

    We were not able to successfully restore a current version of the main vault instance.

    Please continue to use the read-only instance provided for the duration of the weekend.

    Please accept our sincere apologies for the outage.

  • Investigating
    Investigating

    We are currently experiencing an outage with our main vault cluster.

    A read-only copy is available at vault.aws.puzzle.ch

    Write operations are currently not possible. Any entries made on the read-only instance will be overwritten shortly.

GitLab runners
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring
    We implemented a fix and are currently monitoring the result.
  • Identified
    Identified

    Ein Teil der GitLab runner sind im Moment nicht erreichbar. Dies äussert sich mit dieser Fehlermeldung:
    This job is stuck because the project doesn't have any runners online assigned to it.

    Das betrifft alle builds ohne Tags oder Builds mit diesen Tags: openshift-runner, openshift4-runner, medium,small,dagger

OpenShift 4 Monthly Maintenance
  • Completed
    27. May 2024 at 17:3
    Completed
    27. May 2024 at 17:3
    Maintenance has completed successfully.
  • In progress
    27. May 2024 at 16:0
    In progress
    27. May 2024 at 16:0
    Maintenance is now in progress
  • Planned
    27. May 2024 at 16:0
    Planned
    27. May 2024 at 16:0

    As usual we will be conducting our monthly maintenance window to apply updates and configuration enhancements. Services may experience a short interruption and/or periods of degraded performance.

/sys Wartungsfenster
  • Update
    27. May 2024 at 9:3
    Completed
    27. May 2024 at 9:3
    Maintenance has completed successfully.
  • Completed
    23. May 2024 at 20:13
    Completed
    23. May 2024 at 20:13
    Maintenance has completed successfully.
  • In progress
    23. May 2024 at 16:0
    In progress
    23. May 2024 at 16:0
    Maintenance is now in progress
  • Planned
    23. May 2024 at 16:0
    Planned
    23. May 2024 at 16:0

    As usual we will be conducting our monthly maintenance window to apply updates and configuration enhancements. Services may experience a short interruption and/ or periods of degraded performance.

Jenkins ArgoCD migration
  • Completed
    7. May 2024 at 18:22
    Completed
    7. May 2024 at 18:22

    Die Migration wurde wegen technischen Problemen abgebrochen.

  • In progress
    7. May 2024 at 15:30
    In progress
    7. May 2024 at 15:30
    Maintenance is now in progress
  • Planned
    7. May 2024 at 15:30
    Planned
    7. May 2024 at 15:30

    Wir werden unsere Jenkins Instanz während dieser Zeit vom alten auf das neue ArgoCD Setup migrieren. Jenkins wird während diesem Wartungsfenster nicht erreichbar sein.

Apr 2024

Rocket.Chat Upgrade
  • Completed
    30. April 2024 at 15:10
    Completed
    30. April 2024 at 15:10

    The update is complete.
    If your desktop client claims that chat.puzzle.ch is running an outdated version, restart the app a few times.

  • In progress
    30. April 2024 at 15:0
    In progress
    30. April 2024 at 15:0
    Maintenance is now in progress
  • Planned
    30. April 2024 at 15:0
    Planned
    30. April 2024 at 15:0

    Rocket.Chat will be upgraded to 6.7.1.
    Possible downtime is to be expected.

Nextcloud currently not available
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring
    We implemented a fix and are currently monitoring the result.
  • Investigating
    Investigating
    We are currently investigating this incident.
Nextcloud (files.puzzle.ch) seems to be down
  • Update
    Update
    This incident has been resolved.
  • Resolved
    Resolved
    This incident has been resolved.
OpenShift 4 Monthly Maintenance
  • Completed
    22. April 2024 at 17:57
    Completed
    22. April 2024 at 17:57
    Maintenance has completed successfully.
  • In progress
    22. April 2024 at 16:0
    In progress
    22. April 2024 at 16:0
    Maintenance is now in progress
  • Planned
    22. April 2024 at 16:0
    Planned
    22. April 2024 at 16:0

    As usual we will be conducting our monthly maintenance window to apply updates and configuration enhancements. Services may experience a short interruption and/or periods of degraded performance.

Mailserver upgrades
  • Completed
    20. April 2024 at 12:4
    Completed
    20. April 2024 at 12:4
    Maintenance has completed successfully.
  • In progress
    20. April 2024 at 7:0
    In progress
    20. April 2024 at 7:0
    Maintenance is now in progress
  • Planned
    20. April 2024 at 7:0
    Planned
    20. April 2024 at 7:0

    /sys Update 📧: We're upgrading our mail server infrastructure from CentOS 7 to Rocky 9 on Saturday, April 20th. Please be aware of a short downtime lasting about 30 minutes between 09:00am to 1pm. All emails will be processed promptly post-update. Thank you for your patience and inform your customers if applicable!

Apr 2024 to Jun 2024

Next