On-prem platform

This part of the documentation is only intended in the context of a supported PoC (Proof of Concept) together with the Steadybit team. Please, book an appointment to scope your PoC before continuing to evaluate the on-prem solution.

If you just want to try out Steadybit, we recommend you sign up for our SaaS platform.

This page describes some common issues and how to solve them.

Platform and Postgres are in CrashLoopBackOff

• Check the logs of the platform and Postgres containers

    kubectl logs -f -n steadybit-platform steadybit-platform-postgresql-0 --previous
    kubectl logs -f -n steadybit-platform steadybit-platform-0 --previous

• Verify that the Postgres password is correct and base64 encoded in the manifest file

Kubernetes namespaces and deployments show up multiple times in the landscape table

• Check the logs of the agents

    kubectl logs -f -n steadybit-agent steadybit-agent-0

• If you see this error: Missing permissions to create leases for the leader elections or Cannot perform leader election. All agents will behave as leader.

  • Check if the agent has the correct permissions to create leases.

  NAME            APIGROUP↑                    GET            LIST           WATCH          CREATE         PATCH          UPDATE          DELETE          DEL-LIST
  leases          coordination.k8s.io           ✓              ✓              ✓              ✓              ×              ✓               ×               ×

Agents are not able to connect to the platform during an experiment

• Check if you can reach the platform from the agent:

    kubectl exec -it -n steadybit-agent steadybit-agent-0 -- curl -k https://steadybit-platform.steadybit-platform.svc.cluster.local:8080

• Check if the agent can reach the Websocket port of the platform.

  • This is usally port 7878 and can be configured in the platform manifest via environment variable STEADYBIT_WEB_PUBLIC_EXPERIMENT_PORT (helm chart: platform.publicWebsocketPort)

  • If setting the port is not enough, you can set the url via environment variable STEADYBIT_WEB_PUBLIC_EXPERIMENT_URL (helm chart: platform.ingressOrigin)

  • Please also check your ingress configuration.

  spec:
  rules:
    - http:
        paths:
          - pathType: Prefix
            path: /ws
            backend:
              service:
                name: steadybit-platform
                port:
                  number: 7878
          - pathType: Prefix
            path: /
            backend:
              service:
                name: steadybit-platform
                port:
                  number: 80

  • You can try to connect to the websocket port via curl:

  curl 'https://platform.steadybit.com:443/ws' \
    -H 'Upgrade: websocket' \
    -H 'Connection: Upgrade' \
    -H 'Sec-WebSocket-Key: dummy' \
    -H 'Sec-WebSocket-Version: 13' \
    -v --http1.1

  > GET /ws HTTP/1.1
  > Host: platform.steadybit.com
  > User-Agent: curl/8.1.2
  > Accept: */*
  > Upgrade: websocket
  > Connection: Upgrade
  > Sec-WebSocket-Key: dummy
  > Sec-WebSocket-Version: 13
  >
  < HTTP/1.1 101 Switching Protocols
  < Date: Thu, 14 Dec 2023 14:37:03 GMT
  < Connection: upgrade
  < upgrade: websocket

Platform is behind Nginx and the agents are not able to connect to the platform

Error message in the platform logs:

java.lang.IllegalArgumentException: Invalid character found in method name [0x160x030.....[us ]. HTTP method names must be tokens
	at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:407)

Solution:

• set the nginx backend protocol is HTTPS instead of HTTP