Puzzle ITC - Notice history

All systems operational

Operational

OpenShift 4 Cluster cloudscale.ch - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

OpenShift 4 Cluster LPG1 cloudscale.ch - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024
Operational

Puzzle Website (puzzle.ch) - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Mail - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Rocket.Chat - Operational

100% - uptime
Oct 2024 · 99.99%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Taiga - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

PuzzleTime - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Wiki - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Puzzle SSO (Keycloak) - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

CodiMD - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

OpenProject - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Nextcloud - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

PuzzleSkills - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

GitLab - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Jenkins - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Quay (registry.puzzle.ch ) - Operational

100% - uptime
Oct 2024 · 99.88%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Harbor (harbor.puzzle.ch) - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Grafana - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Puzzle News Site (news.puzzle.ch) - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Cryptopus - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Artifactory - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

SonarQube - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Puzzle Meet - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Zammad (help.puzzle.ch) - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Sentry - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

CardDAV - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Webmail - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Argo CD - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Youtrack - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

OKR - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Vault - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Loki - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

LDAP - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Puzzle Docs (docs.puzzle.ch) - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024
Operational

Network Office-Bern - Operational

98% - uptime
Oct 2024 · 100.0%Nov · 94.30%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Google Meet Hardware - Operational

98% - uptime
Oct 2024 · 100.0%Nov · 94.30%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Telephone Setup - Operational

98% - uptime
Oct 2024 · 100.0%Nov · 94.30%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024
Operational

cloudscale.ch - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Amazon AWS - Operational

100% - uptime
Oct 2024 · 100.0%Nov · 100.0%Dec · 100.0%

Previous page

Next page

Oct 2024
Nov 2024
Dec 2024

Notice history

Dec 2024

OpenShift 4 Monthly Maintenance
Scheduled for 23. December 2024 at 17:0 – 20:0 about 3 hours
  • Planned
    23. December 2024 at 17:0
    Planned
    23. December 2024 at 17:0

    As usual we will be conducting our monthly maintenance window to apply updates and configuration enhancements. Services may experience a short interruption and periods of degraded performance.

    RocketChat will be upgraded to the latest version.

    OpenShift will be upgraded to 4.17.

/sys Wartungsfenster
  • Completed
    19. December 2024 at 20:56
    Completed
    19. December 2024 at 20:56

    Our infrastructure maintenance has concluded and all our services are available as usual.

  • In progress
    19. December 2024 at 17:0
    In progress
    19. December 2024 at 17:0
    Maintenance is now in progress
  • Planned
    19. December 2024 at 17:0
    Planned
    19. December 2024 at 17:0

    As usual we will be conducting our monthly maintenance window to apply updates and configuration enhancements. Services may experience a short interruption and/ or periods of degraded performance.

puzzle docs storage maintenance
  • Completed
    17. December 2024 at 11:11
    Completed
    17. December 2024 at 11:11

    DB cleanup successful

  • In progress
    17. December 2024 at 10:55
    In progress
    17. December 2024 at 10:55
    Maintenance is now in progress
  • Planned
    17. December 2024 at 10:55
    Planned
    17. December 2024 at 10:55
    We are planning for a scheduled maintenance during that time.
Puzzle News Deployment migration
  • Completed
    12. December 2024 at 14:57
    Completed
    12. December 2024 at 14:57
    Maintenance has completed successfully.
  • In progress
    9. December 2024 at 9:35
    In progress
    9. December 2024 at 9:35
    We are planning for a scheduled maintenance during that time.
puzzle docs admin and user guide unavailable
  • Resolved
    Resolved

    resolved

  • Investigating
    Investigating
    We are currently investigating this incident.

Nov 2024

/sys Wartungsfenster
  • Completed
    28. November 2024 at 23:0
    Completed
    28. November 2024 at 23:0
    Maintenance has completed successfully
  • In progress
    28. November 2024 at 17:0
    In progress
    28. November 2024 at 17:0
    Maintenance is now in progress
  • Planned
    28. November 2024 at 17:0
    Planned
    28. November 2024 at 17:0

    As usual we will be conducting our monthly maintenance window to apply updates and configuration enhancements. Services may experience a short interruption and/ or periods of degraded performance.

Potential Data loss on Vault
  • Resolved
    Resolved

    Due to brigolaasch of our Lead System Engineer we had to restore a Backup from our Vault. Secrets modified since last night are lost. We are sorry for this inconvenience.

OpenShift 4 Monthly Maintenance
  • Completed
    25. November 2024 at 18:41
    Completed
    25. November 2024 at 18:41

    The upgrade has completed successfully.

  • In progress
    25. November 2024 at 17:0
    In progress
    25. November 2024 at 17:0
    Maintenance is now in progress
  • Planned
    25. November 2024 at 17:0
    Planned
    25. November 2024 at 17:0

    As usual we will be conducting our monthly maintenance window to apply updates and configuration enhancements. Services may experience a short interruption and/or periods of degraded performance.

LDAP Replication recovery
  • Completed
    25. November 2024 at 11:9
    Completed
    25. November 2024 at 11:9
    Maintenance has completed successfully.
  • In progress
    25. November 2024 at 11:0
    In progress
    25. November 2024 at 11:0
    Maintenance is now in progress
  • Planned
    25. November 2024 at 11:0
    Planned
    25. November 2024 at 11:0

    We are in the process of restoring our LDAP replicas following a crash caused by last Thursday's power outage.

Voltage fluctuations on power grid
  • Resolved
    Resolved
    This incident has been resolved.
  • Identified
    Identified

    Our local power supplier is experiencing multiple fluctuations in voltage in their electrical grid. The root cause is with the upstream power provider BKW.

    Most probably due to sudden shifts in the network due to downed power lines because of snow and trees.

    The resulting fluctuations in power grid voltage leads to recurring outages within our infrastructure at the office.

    Some backup and fallback services are inaccessible and will be fixed once power delivery has stabilized for several hours.

    We continue to monitor the situation but we'll not be on site over night.

Oct 2024

/sys Wartungsfenster
  • Completed
    24. October 2024 at 19:0
    Completed
    24. October 2024 at 19:0
    Maintenance has completed successfully.
  • In progress
    24. October 2024 at 16:0
    In progress
    24. October 2024 at 16:0
    Maintenance is now in progress
  • Planned
    24. October 2024 at 16:0
    Planned
    24. October 2024 at 16:0

    As usual we will be conducting our monthly maintenance window to apply updates and configuration enhancements. Services may experience a short interruption and/ or periods of degraded performance.

OpenShift 4 Monthly Maintenance
  • Completed
    21. October 2024 at 18:4
    Completed
    21. October 2024 at 18:4

    The upgrade to 4.16.16 has completed successfully. Rocketchat has been upgraded to the lastest version.

  • In progress
    21. October 2024 at 15:0
    In progress
    21. October 2024 at 15:0
    Maintenance is now in progress
  • Planned
    21. October 2024 at 15:0
    Planned
    21. October 2024 at 15:0

    As usual we will be conducting our monthly maintenance window to apply updates and configuration enhancements. Services may experience a short interruption and/or periods of degraded performance.

We are working on a single node issue in our openshift setup.
  • Resolved
    Resolved

    No further disruptions have occurred. To prevent future outages, we will implement more memory reservations in the next maintenance window.

  • Monitoring
    Monitoring

    The affected node has been rebooted and all services have been restored. We are looking into the root cause and how to mitigate outages in the future.

  • Identified
    Identified

    Some applications might be temporarily unavailable while we redistribute the workload to non-affected parts of the cluster.

Quay Down
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring

    We implemented a hotfix and are monitoring the situation.

  • Identified
    Identified

    We are working on a hotfix.

  • Investigating
    Investigating
    We are currently investigating this incident.
GitLab security update
  • Completed
    10. October 2024 at 10:10
    Completed
    10. October 2024 at 10:10
    Maintenance has completed successfully.
  • In progress
    10. October 2024 at 10:0
    In progress
    10. October 2024 at 10:0
    Maintenance is now in progress.
  • Planned
    9. October 2024 at 22:0
    Planned
    9. October 2024 at 22:0
    We are planning for a scheduled maintenance during that time.

Oct 2024 to Dec 2024

Next