obsctl: A CLI to interact with Observatorium

• Owners:

  • @onprem @saswatamcode

• Related Tickets:

  • https://issues.redhat.com/browse/MON-2199

• Other docs:

  • None

TL;DR#

We can interact with Observatorium API with a convenient and dedicated CLI, which provides capabilities to persist auth data, and allows us to easily & quickly get up and running with our operations.

Pitfalls of Current Solution#

Currently, we do not have a simple and convenient way to interact with the Observatorium API from the command line. Manually crafting cURL commands while authenticating with OIDC or mTLS is not an optimal experience and is generally confusing/trickier for users, both old and new.

Goals#

obsctl will,

• Manage authentication configuration for multiple tenants by saving them locally
• Allow users to switch between tenants conveniently
• Configure and view rules for a tenant
• View series, rules, and labels for a tenant
• Later on, support more such one-off operations for other domains as well (logs, traces, alert-routing configs)

Non-Goals#

• Enabling Observatorium administration via obsctl, for example, adding or removing tenants in Observatorium programmatically or generating operators using rndr

Audience of this proposal#

Observatorium Devs and Users.

How#

As obsctl is a CLI, authentication options (OIDC) need to be persisted across sessions. obsctl aims to achieve this by saving those configurations locally at a configuration directory and maintaining a “current” tenant & a “current” instance of Observatorium, which we can refer to as “context”.

The “current context” will basically represent the logged-in tenant and the Observatorium API endpoint which the tenant is allowed to use. All operations will be done based on this “current context” configuration. In case, the “current context” is empty, i.e, there is no tenant currently logged-in or no API provided, obsctl will error out.

obsctl will also allow the user to switch from the “current context” to another one. This can be achieved by creating a map of API configurations, each having tenants, that are uniquely referenceable.

For OIDC/OAuth2 based authentication, obsctl will try to fetch access token from SSO service (if supported) and store it locally, and automatically refresh the access token if the locally saved token expires.

Other auth flows such as OIDC Device Flow or mTLS will be added as optional flags, so that users can use flows of their choice.

The configuration will be saved at a path like os.UserConfigDir()/obsctl/config.json. Here os.UserConfigDir is used to keep obsctl consistent across platforms.

Authentication Commands to be supported by obsctl#

• login --tenant --api <name/URL> <OIDC flags> : Login as a tenant and save their configuration locally.
  • --tenant: The name used to refer to the tenant.
  • --api-name: The name used to refer to the api.
• context api add --url <URL> --name <api name> : Add a referrable instance of Observatorium. If name is not provided then URL hostname is used.
• context switch <api name>/<tenant>: Switch from one context to another.
• context list: List all the possible contexts.
• context current: Display the “current” context.

Initially, obsctl aims to support one-off metrics-based operations for users. Endpoints for other signals can be added later on.

Metric-based Commands to be supported by obsctl#

• metrics get rules.raw: Get configured Rules for a tenant (YAML, hits api/v1/rules/raw).
• metrics get rules <URL params as flags>: Read rules for a tenant. (JSON, hits api/v1/rules).
• metrics get series <URL params as flags>: Read series for a tenant.
• metrics get labels <URL params as flags>: Read labels for a tenant.
• metrics get labelvalues <URL params as flags>: Read label values for a tenant.
• metrics set --rule.file=<rules-config>: Set Rules for a tenant by passing in Rule file.
• metrics query <PromQL query>: Query data for a particular tenant (JSON).

Additionally, obsctl will also provide statistical information (such as time taken, response size) about the performed operations along with the result, which might be useful for users willing to debug their instances of Observatorium.

Alternatives#

• Continuing with the existing process of manually crafting cURL requests and documenting that in detail.