Apply Red Hat Quay configuration
Learn how Red Hat Quay configuration works across deployment types, including the role of config.yaml and common configuration concepts.
Getting started with Project Quay configuration
Project Quay is a secure artifact registry that can be deployed as a self-managed installation or through the Red Hat Quay on OpenShift Container Platform Operator, and then configured by adjusting settings to meet the needs of your environment.
Each deployment type offers a different approach to configuration and management, but each relies on the same set of configuration parameters to control registry behavior. Common configuration parameters allow administrators to define how their registry interacts with users, storage backends, authentication providers, security policies, and other integrated services.
You can configure Project Quay in one of two ways, depending on your deployment type:
-
On-premises Project Quay: With an on-premises Project Quay deployment, a registry administrator provides a
config.yamlfile that includes all required parameters. For this deployment type, the registry is unable to start without a valid configuration. -
Project Quay Operator: By default, the Project Quay Operator automatically configures your Project Quay deployment by generating the minimal required values and deploying the necessary components for you. After the initial deployment, you can customize your registry’s behavior by modifying the
QuayRegistrycustom resource, or by using the OpenShift Container Platform Web Console.
This guide offers an overview of the following configuration concepts:
-
How to retrieve, inspect, and modify your current configuration for both on-premises and Operator-based Project Quay deployment types.
-
The minimal configuration fields required for startup.
-
An overview of all available Project Quay configuration fields and YAML examples for those fields.
Project Quay configuration disclaimer
Some Project Quay configuration parameters and feature flags are undocumented or not actively supported. Modifying these settings can cause unexpected behavior in your deployment, so you should use them only with caution.
In both self-managed and Operator-based deployments of Project Quay, certain features and configuration parameters are not actively used or implemented. As a result, some feature flags, such as those that enable or disable specific functionality, or configuration parameters that are not explicitly documented or supported by or requested for documentation by Red Hat Support, should only be modified with caution.
Understanding the Project Quay configuration file
The Project Quay config.yaml file defines registry behavior.
The config.yaml file must include all required configuration fields for the registry to start. Project Quay administrators can also define optional parameters that customize their registry, such as authentication parameters, storage parameters, proxy cache parameters, and so on.
The config.yaml file must be written using valid YAML ("YAML Ain’t Markup Language") syntax, and Project Quay cannot start if the file itself contains any formatting errors or missing required fields. Regardless of deployment type, whether on-premises or Red Hat Quay on OpenShift Container Platform with the Operator, the YAML principles stay the same, even if the required configuration fields are slightly different.
The following section outlines basic YAML syntax relevant to creating and editing the Project Quay config.yaml file. For a more complete overview of YAML, see "What is YAML?".
Key-value pairs
Configuration fields within a config.yaml file are written as key-value pairs in the following form:
# ...
EXAMPLE_FIELD_NAME: <value>
# ...
The # … comment denotes fields before and after this specific field. Note that by supplying the #, or hash symbol, comments can be provided within the YAML file.
Each line within a config.yaml file contains a field name, followed by a colon, a space, and then an appropriate value that matches with the key. The following example shows you how the AUTHENTICATION_TYPE configuration field must be formatted in your config.yaml file.
AUTHENTICATION_TYPE: Database
# ...
AUTHENTICATION_TYPE-
Specifies the authentication engine to use for credential authentication.
In the previous example, the AUTHENTICATION_TYPE is set to Database, however, different deployment types require a different value. The following example shows you how your config.yaml file might look if LDAP, or Lightweight Directory Access Protocol, was used for authentication:
AUTHENTICATION_TYPE: LDAP
# ...
Indentation and nesting
Many Project Quay configuration fields require indentation to indicate nested structures. Indentation must be done by using white spaces, or literal space characters; tab characters are not allowed by design. Indentation must be consistent across the file. The following YAML snippet shows you how the BUILDLOGS_REDIS field uses indentation for the required host, password, and port fields:
# ...
BUILDLOGS_REDIS:
host: quay-server.example.com
password: example-password
port: 6379
# ...
Lists
In some cases, the Project Quay configuration field relies on lists to define certain values. You format lists by using a hyphen (-) followed by a space. The following example shows you how the SUPER_USERS configuration field uses a list to define superusers:
# ...
SUPER_USERS:
- quayadmin
# ...
Quoted values
Some Project Quay configuration fields require the use of quotation marks ("") to properly define a variable. This is generally not required. The following examples shows you how the FOOTER_LINKS configuration field uses quotation marks to define the TERMS_OF_SERVICE_URL, PRIVACY_POLICY_URL, SECURITY_URL, and ABOUT_URL:
FOOTER_LINKS:
"TERMS_OF_SERVICE_URL": "https://www.index.hr"
"PRIVACY_POLICY_URL": "https://www.jutarnji.hr"
"SECURITY_URL": "https://www.bug.hr"
"ABOUT_URL": "https://www.zagreb.hr"
Comments
The hash symbol, or #, can be placed at the beginning of a line to add comments or to temporarily disable a configuration field. The configuration parser ignores them, so they do not affect registry behavior. For example:
# ...
# FEATURE_UI_V2: true
# ...
In this example, the configuration parser ignores the FEATURE_UI_V2 configuration, meaning that the option to use the v2 UI is disabled. Using the # symbol on a required configuration field results in failure for the registry to start.
Configure a standalone Red Hat Quay deployment
Configure a standalone or on-premises Red Hat Quay deployment using minimal config.yaml examples, post-deployment updates, and procedures to verify, troubleshoot, and read the configuration file.
On-premise Project Quay configuration overview
For on-premise Project Quay deployments, you manage a config.yaml file that Project Quay reads at container startup. You must restart the registry container after you change the file because Project Quay does not reload configuration dynamically.
This chapter provides an overview of the following concepts:
-
The minimal required configuration fields.
-
How to edit and manage your configuration after deployment.
This section applies specifically to on-premise Project Quay deployment types. For information about configuring Red Hat Quay on OpenShift Container Platform, see "Red Hat Quay on OpenShift Container Platform configuration overview".
Minimal required fields for standalone config
The following configuration fields are required to start an on-premises Project Quay deployment. You must include each field in your config.yaml file before the registry can start.
| Field | Type | Description |
|---|---|---|
AUTHENTICATION_TYPE (Required) |
String |
The authentication engine to use for credential authentication. Values: One of |
BUILDLOGS_REDIS (Required) |
Object |
Redis connection details for build logs caching. |
.host (Required) |
String |
The hostname at which Redis is accessible. |
.password |
String |
The password to connect to the Redis instance. |
DATABASE_SECRET_KEY (Required) |
String |
Key used to encrypt sensitive fields within the database. This value should never be changed once set, otherwise all reliant fields, for example, repository mirror username and password configurations, are invalidated. This value is set automatically by the Project Quay Operator for Operator-based deployments. For standalone deployments, administrators can provide their own key using Open SSL or a similar tool. Key length should not exceed 63 characters. |
DB_URI (Required) |
String |
The URI for accessing the database, including any credentials. |
DISTRIBUTED_STORAGE_CONFIG (Required) |
Object |
Configuration for storage engine(s) to use in Project Quay. Each key represents an unique identifier for a storage engine. The value consists of a tuple of (key, value) forming an object describing the storage engine parameters. Default: |
SECRET_KEY (Required) |
String |
Key used to encrypt the session cookie and the CSRF token needed for correct interpretation of the user session. The value should not be changed when set. Should be persistent across all Project Quay instances. If not persistent across all instances, login failures and other errors related to session persistence might occur. |
SERVER_HOSTNAME (Required) |
String |
The URL at which Project Quay is accessible, without the scheme. |
SETUP_COMPLETE (Required) |
Boolean |
This is an artifact left over from earlier versions of the software and currently it must be specified with a value of |
USER_EVENTS_REDIS (Required) |
Object |
Redis connection details for user event handling. |
.host (Required) |
String |
The hostname at which Redis is accessible. |
.port (Required) |
Number |
The port at which Redis is accessible. |
.password |
String |
The password to connect to the Redis instance. |
Minimal configuration file examples
You can use minimal config.yaml file examples to start an on-premises Project Quay registry with local or cloud-based storage. These examples show only the required fields.
This section provides two examples of a minimal configuration file: one example that uses local storage, and another example that uses cloud-based storage with Google Cloud Platform.
Minimal configuration using local storage
A minimal config.yaml for on-premises Project Quay uses local storage for images. Use this example only for proof of concept deployments, not production.
|
Important
|
Only use local storage when deploying a registry for proof of concept purposes. Local storage is not intended for production purposes. When using local storage, you must map the registry to a local directory to the |
AUTHENTICATION_TYPE: Database
BUILDLOGS_REDIS:
host: <quay-server.example.com>
password: <password>
port: <port>
DATABASE_SECRET_KEY: <example_database_secret_key>
DB_URI: postgresql://<username>:<password>@<registry_url>.com:<port>/quay
DISTRIBUTED_STORAGE_CONFIG:
default:
- LocalStorage
- storage_path: /datastorage/registry
SECRET_KEY: <example_secret_key>
SERVER_HOSTNAME: <server_host_name>
SETUP_COMPLETE: true
USER_EVENTS_REDIS:
host: <redis_events_url>
password: <password>
port: <port>
Minimal configuration using cloud-based storage
A minimal config.yaml for on-premises Project Quay can use cloud-based object storage such as Google Cloud Platform. Use this pattern when you deploy Project Quay with a supported enterprise storage backend.
In most production environments, Project Quay administrators use cloud or enterprise-grade storage backends provided by supported vendors. The following example shows you how to configure Project Quay to use Google Cloud Platform for image storage. For a complete list of supported storage providers, see "Image storage".
|
Note
|
When using a cloud or enterprise-grade storage backend, additional configuration, such as mapping the registry to a local directory, is not required. |
AUTHENTICATION_TYPE: Database
BUILDLOGS_REDIS:
host: <quay-server.example.com>
password: <password>
port: <port>
DATABASE_SECRET_KEY: <example_database_secret_key>
DB_URI: postgresql://<username>:<password>@<registry_url>.com:<port>/quay
DISTRIBUTED_STORAGE_CONFIG:
default:
- GoogleCloudStorage
- access_key: <access_key>
bucket_name: <bucket_name>
secret_key: <secret_key>
storage_path: /datastorage/registry
DISTRIBUTED_STORAGE_DEFAULT_LOCATIONS: []
DISTRIBUTED_STORAGE_PREFERENCE:
- default
SECRET_KEY: <example_secret_key>
SERVER_HOSTNAME: <server_host_name>
SETUP_COMPLETE: true
USER_EVENTS_REDIS:
host: <redis_events_url>
password: <password>
port: <port>
Modifying your configuration file after deployment
To update your on-premises Project Quay configuration after deployment, you can edit the config.yaml file and restart the quay-registry container. You can retrieve the file from the container if you do not have direct access to it.
After deploying a Project Quay registry with an initial config.yaml file, Project Quay administrators can update the configuration file to enable or disable features as needed. This flexibility allows administrators to tailor the registry to fit their specific environment needs, or to meet certain security policies.
|
Note
|
Because the |
The following procedure shows you how to retrieve the config.yaml file from the quay-registry container, how to enable a new feature by adding that feature’s configuration field to the file, and how to restart the quay-registry container using Podman.
-
You have deployed Project Quay.
-
You are a registry administrator.
-
If you have access to the
config.yamlfile:-
Navigate to the directory that is storing the
config.yamlfile. For example:$ cd /home/<username>/<quay-deployment-directory>/config -
Make changes to the
config.yamlfile by adding a new feature flag. The following example enables the v2 UI:# ... FEATURE_UI_V2: true # ... -
Save the changes made to the
config.yamlfile. -
Restart the
quay-registrypod by entering the following command:$ podman restart <container_id>
-
-
If you do not have access to the
config.yamlfile and need to create a new file while keeping the same credentials:-
Retrieve the container ID of your
quay-registrypod by entering the following command:$ podman psExample outputCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 5f2297ef53ff registry.redhat.io/rhel8/postgresql-13:1-109 run-postgresql 20 hours ago Up 20 hours 0.0.0.0:5432->5432/tcp postgresql-quay 3b40fb83bead registry.redhat.io/rhel8/redis-5:1 run-redis 20 hours ago Up 20 hours 0.0.0.0:6379->6379/tcp redis 0b4b8fbfca6d registry-proxy.engineering.redhat.com/rh-osbs/quay-quay-rhel8:v3.14.0-14 registry 20 hours ago Up 20 hours 0.0.0.0:80->8080/tcp, 0.0.0.0:443->8443/tcp, 7443/tcp, 9091/tcp, 55443/tcp quay -
Copy the
config.yamlfile from thequay-registrypod to a directory by entering the following command:$ podman cp <container_id>:/quay-registry/conf/stack/config.yaml ./config.yaml -
Make changes to the
config.yamlfile by adding a new feature flag. The following example sets theAUTHENTICATION_TYPEtoLDAP# ... AUTHENTICATION_TYPE: LDAP # ... -
Re-deploy the registry, mounting the
config.yamlfile into thequay-registryconfiguration volume by entering the following command:$ sudo podman run -d --rm -p 80:8080 -p 443:8443 \ --name=quay \ -v /home/<username>/<quay-deployment-directory>/config:/conf/stack:Z \ registry.redhat.io/quay/quay-rhel8:v3.14.0
-
Troubleshooting the configuration file for standalone deployments
To identify configuration errors that prevent your on-premises Project Quay registry from starting, you can deploy the quay-registry container interactively and review the validation output.
Failure to add all of the required configuration field, or to provide the proper information for some parameters, might result in the quay-registry container failing to deploy. Use the following procedure to view and troubleshoot a failed on-premises deployment type.
-
You have created a minimal configuration file.
-
Attempt to deploy the
quay-registrycontainer by entering the following command. Note that this command uses the-it, which shows you debugging information:$ podman run -it --rm -p 80:8080 -p 443:8443 --name=quay -v /home/<username>/<quay-deployment-directory>/config:/conf/stack:Z -v /home/<username>/<quay-deployment-directory>/storage:/datastorage:Z 33f1c3dc86beExample output--- +------------------------+-------+--------+ | LDAP | - | X | +------------------------+-------+--------+ | LDAP_ADMIN_DN is required | X | +-----------------------------------------+ | LDAP_ADMIN_PSSWD is required | X | +-----------------------------------------+ | . . . Connection refused | X | +-----------------------------------------+ ---In this example, the
quay-registrycontainer failed to deploy because improper LDAP credentials were provided.
Reading the configuration file from a standalone deployment by using Podman
To obtain configuration information for your Project Quay deployment and troubleshoot issues, you can use podman cp or podman exec for standalone deployments. You can then update your config.yaml file, search the Red Hat Knowledgebase, or file a support ticket.
-
To obtain configuration information on standalone Project Quay deployments, you can use
podman cporpodman exec.-
To use the
podman copycommand, enter the following commands:$ podman cp <quay_container_id>:/conf/stack/config.yaml /tmp/local_directory/To display this information in your terminal, enter the following command:
$ cat /tmp/local_directory/config.yaml -
To use
podman exec, enter the following commands:$ podman exec -it <quay_container_id> cat /conf/stack/config.yamlExample outputBROWSER_API_CALLS_XHR_ONLY: false ALLOWED_OCI_ARTIFACT_TYPES: application/vnd.oci.image.config.v1+json: - application/vnd.oci.image.layer.v1.tar+zstd application/vnd.sylabs.sif.config.v1+json: - application/vnd.sylabs.sif.layer.v1+tar AUTHENTICATION_TYPE: Database AVATAR_KIND: local BUILDLOGS_REDIS: host: quay-server.example.com password: strongpassword port: 6379 DATABASE_SECRET_KEY: 05ee6382-24a6-43c0-b30f-849c8a0f7260 DB_CONNECTION_ARGS: {} ---
-
Manage Red Hat Quay configuration on OpenShift Container Platform
Learn how the Quay Operator manages registry configuration on OpenShift through the QuayRegistry custom resource, managed and unmanaged components, and the config bundle Secret.
How Operator configuration works
When deploying Red Hat Quay on OpenShift Container Platform, the registry configuration is managed declaratively through two primary mechanisms: the QuayRegistry custom resource (CR) and the configBundleSecret resource. You use these mechanisms to configure and manage your registry deployment.
Understanding the QuayRegistry CR
The QuayRegistry CR is used to determine whether a component is managed, or automatically handled by the Operator, or unmanaged, or provided externally by the user.
By default, the QuayRegistry CR contains the following key fields:
-
configBundleSecret: The name of a Kubernetes Secret containing theconfig.yamlfile which defines additional configuration parameters. -
name: The name of your Project Quay registry. -
namespace: The namespace, or project, in which the registry was created. -
spec.components: A list of components that the Operator automatically manages. Each component entry includes the following fields:-
kind: The name of the component -
managed: A boolean that addresses whether the component lifecycle is handled by the Project Quay Operator. Settingmanaged: trueto a component in theQuayRegistryCR means that the Operator manages the component. -
secretRef: Optional. For thetlscomponent only, references an externalkubernetes.io/tlsSecret whenmanagedisfalse. For more information, see Referencing an external TLS Secret for Red Hat Quay on OpenShift Container Platform.
-
All QuayRegistry components are automatically managed and auto-filled upon reconciliation for visibility unless specified otherwise. The following sections highlight the major QuayRegistry components and provide an example YAML file that shows the default settings.
Managed components
Managed components are Project Quay registry components that the Operator automatically configures and installs. By using managed components, you simplify deployment and reduce manual configuration tasks.
| Field | Type | Description |
|---|---|---|
|
Boolean |
Holds overrides for deployment of Red Hat Quay on OpenShift Container Platform, such as environment variables and number of replicas. This component cannot be set to unmanaged ( |
|
Boolean |
Used for storing registry metadata. Currently, PostgreSQL version 13 is used. |
|
Boolean |
Provides image vulnerability scanning. You can override ephemeral scratch storage for image layer extraction by using |
|
Boolean |
Stores live builder logs and the locking mechanism that is required for garbage collection. You can override CPU and memory resources for this component when it is managed. |
|
Boolean |
Adjusts the number of |
|
Boolean |
Stores image layer blobs. When set to |
|
Boolean |
Provides an external entrypoint to the Project Quay registry from outside of OpenShift Container Platform. |
|
Boolean |
Configures repository mirror workers to support optional repository mirroring. |
|
Boolean |
Features include a Grafana dashboard, access to individual metrics, and notifications for frequently restarting |
|
Boolean |
Configures whether SSL/TLS is automatically handled. When |
|
Boolean |
Configures a managed Clair database. This is a separate database than the PostgreSQL database that is used to deploy Project Quay. |
The following example shows you the default configuration for the QuayRegistry custom resource provided by the Project Quay Operator. It is available on the OpenShift Container Platform web console.
apiVersion: quay.redhat.com/v1
kind: QuayRegistry
metadata:
name: <example_registry>
namespace: <namespace>
spec:
configBundleSecret: config-bundle-secret
components:
- kind: quay
managed: true
- kind: postgres
managed: true
- kind: clair
managed: true
- kind: redis
managed: true
- kind: horizontalpodautoscaler
managed: true
- kind: objectstorage
managed: true
- kind: route
managed: true
- kind: mirror
managed: true
- kind: monitoring
managed: true
- kind: tls
managed: true
- kind: clairpostgres
managed: true
Disabling the monitoring component
Disabling the monitoring component sets the monitoring component to unmanaged in the QuayRegistry custom resource. You must disable monitoring when you install the Project Quay Operator in a single namespace, or you can disable it in multi-namespace installations to use your own monitoring stack.
|
Note
|
Monitoring cannot be enabled when the Project Quay Operator is installed in a single namespace. You might also disable monitoring in multi-namespace deployments if you use an external Prometheus or Grafana instance, want to reduce resource overhead, or require custom observability integration. |
apiVersion: quay.redhat.com/v1
kind: QuayRegistry
metadata:
name: example-registry
namespace: quay-enterprise
spec:
components:
- kind: monitoring
managed: false
Disabling the mirroring component
Repository mirroring in Project Quay allows you to automatically synchronize container images from remote registries into your local Project Quay instance. The Project Quay Operator deploys a separate mirroring worker component that handles these synchronization tasks.
You can disable the managed mirroring component by setting it to managed: false in the QuayRegistry custom resource.
|
Note
|
Disabling managed mirroring means that the Operator does not deploy or reconcile any mirroring pods. You are responsible for creating, scheduling, and maintaining mirroring jobs manually. For most production deployments, leaving mirroring as |
apiVersion: quay.redhat.com/v1
kind: QuayRegistry
metadata:
name: example-registry
namespace: quay-enterprise
spec:
components:
- kind: mirroring
managed: false
Using unmanaged components for dependencies
Unmanaged components are Project Quay dependencies such as PostgreSQL, Redis, or object storage that you deploy and maintain outside of the Operator’s control. You use unmanaged components to integrate existing infrastructure or meet specific configuration requirements.
|
Note
|
If you are using an unmanaged PostgreSQL database, and the version is PostgreSQL 10, it is highly recommended that you upgrade to PostgreSQL 13. PostgreSQL 10 had its final release on November 10, 2022 and is no longer supported. For more information, see the PostgreSQL Versioning Policy. |
For more information about configuring unmanaged components, see "Configure Red Hat Quay database and Redis backends", "Object storage backend configuration fields", "Configure networking for Red Hat Quay", and "Tune Operator autoscaling and component resources".
Referencing an external TLS Secret for Red Hat Quay on OpenShift Container Platform
Reference an external kubernetes.io/tls Secret from the tls component of the QuayRegistry custom resource (CR) to enable automated certificate rotation from sources such as cert-manager, HashiCorp Vault, or manual Secret updates.
With this approach, the Project Quay Operator uses the certificate and private key from the referenced Secret instead of embedding ssl.cert and ssl.key files in the configBundleSecret resource.
When you use an external TLS Secret, set spec.components[kind: tls].managed to false and specify secretRef. The Operator watches the referenced Secret and performs a rolling restart of Project Quay pods when the certificate data changes.
|
Important
|
|
-
You have deployed the Project Quay Operator and a
QuayRegistryCR. -
You have a TLS Secret of type
kubernetes.io/tlsin the same namespace as theQuayRegistry, withtls.crtandtls.keydata keys. -
The certificate and private key are valid (format, key match, chain, and hostname).
-
Create a TLS Secret in the registry namespace, for example:
apiVersion: v1 kind: Secret metadata: name: my-quay-tls namespace: <namespace> type: kubernetes.io/tls data: tls.crt: <base64_encoded_certificate> tls.key: <base64_encoded_private_key> -
Set the
tlscomponent to unmanaged and reference the Secret by entering the following command:$ oc patch quayregistry <registry_name> -n <namespace> --type=merge -p '{"spec":{"components":[{"kind":"tls","managed":false,"secretRef":{"name":"my-quay-tls"}}]}}' -
Verify that the
QuayRegistryCR contains the expected configuration:$ oc get quayregistry <registry_name> -n <namespace> -o yamlExample outputspec: components: - kind: tls managed: false secretRef: name: my-quay-tls -
Wait for the Operator to reconcile the registry. When certificate data in the referenced Secret changes, the Operator triggers a rolling restart of Project Quay pods to load the updated certificate.
-
Verify that TLS from the external Secret is ready by entering the following command. When the TLS component is healthy, the
ComponentTLSReadycondition reportsstatus: "True".$ oc wait quayregistry <registry_name> -n <namespace> --for=condition=ComponentTLSReady=True --timeout=300sYou can also review all component conditions in the OpenShift Container Platform web console on the QuayRegistry details page, or by entering
oc get quayregistry <registry_name> -n <namespace> -o yamland checkingstatus.conditions.
Modifying the QuayRegistry CR after deployment
Modifying the QuayRegistry custom resource (CR) in Project Quay after deployment lets you customize or reconfigure aspects of your Project Quay environment.
Project Quay administrators might modify the QuayRegistry CR for the following reasons:
-
To change component management: Switch components from
managed: truetomanaged: falsein order to bring your own infrastructure. For example, you might setkind: objectstorageto unmanaged to integrate external object storage platforms such as Google Cloud Storage or Nutanix. -
To apply custom configuration: Update or replace the
configBundleSecretto apply new configuration settings, for example, authentication providers, external SSL/TLS settings, feature flags. -
To enable or disable features: Toggle features like repository mirroring, Clair scanning, or horizontal pod autoscaling by modifying the
spec.componentslist. -
To scale the deployment: Adjust environment variables or replica counts for the Quay application.
-
To integrate with external services: Provide configuration for external PostgreSQL, Redis, or Clair databases, and update endpoints or credentials.
Modifying the QuayRegistry CR by using the OpenShift Container Platform web console
To modify the QuayRegistry custom resource in Project Quay, you can use the OpenShift Container Platform web console to change component management settings. You can set managed components to unmanaged and use your own infrastructure.
-
You are logged into OpenShift Container Platform as a user with admin privileges.
-
You have installed the Project Quay Operator.
-
On the OpenShift Container Platform web console, click Operators → Installed Operators.
-
Click Red Hat Quay.
-
Click Quay Registry.
-
Click the name of your Project Quay registry, for example, example-registry.
-
Click YAML.
-
Adjust the
managedfield of the desired component to eitherTrueorFalse. -
Click Save.
NoteSetting a component to unmanaged (
managed: false) might require additional configuration. For more information about setting unmanaged components in theQuayRegistryCR, see Using unmanaged components for dependencies.
Modifying the QuayRegistry CR by using the CLI
To modify the QuayRegistry custom resource in Project Quay, you can use the CLI to change component management settings. You can set managed components to unmanaged and use your own infrastructure.
-
You are logged in to your OpenShift Container Platform cluster as a user with admin privileges.
-
Edit the
QuayRegistryCR by entering the following command:$ oc edit quayregistry <registry_name> -n <namespace> -
Make the desired changes to the
QuayRegistryCR.NoteSetting a component to unmanaged (
managed: false) might require additional configuration. For more information about setting unmanaged components in theQuayRegistryCR, see Using unmanaged components for dependencies. -
Save the changes.
Understanding the configBundleSecret resource
The configBundleSecret resource is a Kubernetes Secret that stores the config.yaml file for Project Quay. You use this secret to configure authentication backends, feature flags, TLS settings, and other registry parameters that the Operator merges with managed component settings.
Project Quay administrators might update this secret for the following reasons:
-
Enable a new authentication method
-
Add custom SSL/TLS certificates
-
Enable features
-
Modify security scanning settings
If this field is omitted, the Project Quay Operator automatically generates a configuration secret based on default values and managed component settings. If you provide this field, Project Quay uses the config.yaml contents as the base configuration and merges them with values from managed components to form the final configuration, which is mounted into the quay application pods.
Modifying the configuration file by using the OpenShift Container Platform web console
To modify the config.yaml file stored in the configBundleSecret, you can use the OpenShift Container Platform web console to edit the secret and add configuration key-value pairs.
-
You are logged in to the OpenShift Container Platform cluster as a user with admin privileges.
-
On the OpenShift Container Platform web console, click Operators → Installed Operators → Red Hat Quay.
-
Click Quay Registry.
-
Click the name of your Project Quay registry, for example, example-registry.
-
On the QuayRegistry details page, click the name of your Config Bundle Secret, for example, example-registry-config-bundle.
-
Click Actions → Edit Secret.
-
In the Value box, add the desired key/value pair. For example, to add a superuser to your Red Hat Quay on OpenShift Container Platform deployment, add the following reference:
SUPER_USERS: - quayadmin -
Click Save.
NoteYou must base64 encode any updated
config.yamlfile before placing it in the Secret. Ensure the Secret name matches the value specified inspec.configBundleSecretresource. Once the Secret is updated, the Operator detects the change and automatically rolls out updates to the Project Quay pods.For detailed steps, see "Updating configuration secrets through the Project Quay UI."
-
Verify that the changes have been accepted:
-
On the OpenShift Container Platform web console, click Operators → Installed Operators → Red Hat Quay.
-
Click Quay Registry.
-
Click the name of your Project Quay registry, for example, example-registry.
-
Click Events. If successful, the following message is displayed:
All objects created/updated successfully
-
Modifying the configuration file by using the CLI
To modify the config.yaml file for your Project Quay registry and enable new features, you can download the existing configuration from the configBundleSecret by using the CLI. After making changes, you can re-upload the configBundleSecret resource to apply the changes.
|
Note
|
Modifying the |
-
You are logged in to the OpenShift Container Platform cluster as a user with admin privileges.
-
Describe the
QuayRegistryresource by entering the following command:$ oc describe quayregistry -n <quay_namespace># ... Config Bundle Secret: example-registry-config-bundle-v123x # ... -
Obtain the secret data by entering the following command:
$ oc get secret -n <quay_namespace> <example-registry-config-bundle-v123x> -o jsonpath='{.data}'{ "config.yaml": "RkVBVFVSRV9VU0 ... MDAwMAo=" } -
Decode the data into a YAML file into the current directory by passing in the
>> config.yamlflag. For example:$ echo 'RkVBVFVSRV9VU0 ... MDAwMAo=' | base64 --decode >> config.yaml -
Make the desired changes to your
config.yamlfile, and then save the file asconfig.yaml. -
Create a new
configBundleSecretYAML by entering the following command.$ touch <new_configBundleSecret_name>.yaml -
Create the new
configBundleSecretresource, passing in theconfig.yamlfile` by entering the following command:$ oc -n <namespace> create secret generic <secret_name> \ --from-file=config.yaml=</path/to/config.yaml> \ --dry-run=client -o yaml > <new_configBundleSecret_name>.yamlwhere:
- </path/to/config.yaml>
-
Specifies your base64 decoded
config.yamlfile.
-
Create the
configBundleSecretresource by entering the following command:$ oc create -n <namespace> -f <new_configBundleSecret_name>.yamlsecret/config-bundle created -
Update the
QuayRegistryYAML file to reference the newconfigBundleSecretobject by entering the following command:$ oc patch quayregistry <registry_name> -n <namespace> --type=merge -p '{"spec":{"configBundleSecret":"<new_configBundleSecret_name>"}}'quayregistry.quay.redhat.com/example-registry patched
-
Verify that the
QuayRegistryCR has been updated with the newconfigBundleSecret:$ oc describe quayregistry -n <quay_namespace># ... Config Bundle Secret: <new_configBundleSecret_name> # ...After patching the registry, the Project Quay Operator automatically reconciles the changes.
Reading the configuration file from an Operator deployment by using the OpenShift CLI
To obtain configuration information for your Project Quay deployment and troubleshoot issues, you can use oc exec, oc cp, or oc rsync for Operator deployments. Use these options to update your config.yaml file, search the Red Hat Knowledgebase, or file a support ticket.
-
To obtain configuration information on Project Quay Operator deployments, you can use
oc exec,oc cp, oroc rsync.-
To use the
oc execcommand, enter the following command:$ oc exec -it <quay_pod_name> -- cat /conf/stack/config.yamlThis command returns your
config.yamlfile directly to your terminal. -
To use the
oc copycommand, enter the following commands:$ oc cp <quay_pod_name>:/conf/stack/config.yaml /tmp/config.yamlTo display this information in your terminal, enter the following command:
$ cat /tmp/config.yaml -
To use the
oc rsynccommand, enter the following commands:oc rsync <quay_pod_name>:/conf/stack/ /tmp/local_directory/To display this information in your terminal, enter the following command:
$ cat /tmp/local_directory/config.yamlExample outputDISTRIBUTED_STORAGE_CONFIG: local_us: - RHOCSStorage - access_key: redacted bucket_name: lht-quay-datastore-68fff7b8-1b5e-46aa-8110-c4b7ead781f5 hostname: s3.openshift-storage.svc.cluster.local is_secure: true port: 443 secret_key: redacted storage_path: /datastorage/registry DISTRIBUTED_STORAGE_DEFAULT_LOCATIONS: - local_us DISTRIBUTED_STORAGE_PREFERENCE: - local_us
-
Retrieve Red Hat Quay configuration by using the API
Retrieve active Red Hat Quay configuration settings by using the superuser API.
Retrieving active configuration settings by using the API
To retrieve Project Quay configuration settings from the command line, you can enable FEATURE_SUPERUSER_CONFIGDUMP and call the v1/superuser/config API endpoint with a superuser OAuth 2 access token.
As a Project Quay superuser, you can return all Flask configuration fields that are set, which you can use to show proof of compliance for various security policies, such as PCI-DSS 4.0.
-
You have set
FEATURE_SUPERUSER_CONFIGDUMP: truein yourconfig.yamlfile. -
You have assigned the superuser role to a user in your
config.yamlfile. -
You have generated an OAuth 2 access token for the superuser.
-
Retrieve configuration settings by using the
v1/superuser/configAPI endpoint. For example:$ curl -X GET -H "Authorization: Bearer <bearer_token>" "http://<quay-server.example.com>/api/v1/superuser/config" | jq -r .configExample output... "TEAM_RESYNC_STALE_TIME": "30m", "UI_DELAY_AFTER_WRITE_SECONDS": 3, "UI_MODELCARD_ANNOTATION": {}, "UI_MODELCARD_ARTIFACT_TYPE": "application/x-mlmodel", "UI_MODELCARD_LAYER_ANNOTATION": { "org.opencontainers.image.title": "README.md" } ... -
You can pass in one of
.config,.env,.warning, or.schemato return specific information. For example:$ curl -X GET -H "Authorization: Bearer <bearer_token>" "http://<quay-server.example.com>/api/v1/superuser/config" | jq -r .warningExample output... "BILLING_TYPE": "FakeStripe", "BUILDLOGS_OPTIONS": [], "BUILD_MANAGER": null, "CDN_SPECIFIC_NAMESPACES": [], "CHANNEL_COLORS": [ ] ...
Configure database and Redis backends
Configure Project Quay to use an external PostgreSQL database and a Redis instance. This job covers unmanaged database setup, external PostgreSQL integration, and unmanaged Redis configuration.
Using an external PostgreSQL database
Using an external PostgreSQL database with Project Quay lets you manage your own database infrastructure instead of using the Operator-managed database. You must ensure that required configuration and extensions, such as pg_trgm, are in place before deployment.
If you use the Operator-managed PostgreSQL database and require encryption in transit, you can enable TLS on the postgres component without switching to an external database. For more information, see "TLS encryption for Operator-managed PostgreSQL".
|
Important
|
Do not share the same PostgreSQL database between Project Quay and Clair deployments. Each service must use its own database instance. Sharing databases with other workloads is also not supported, because connection-intensive components such as Project Quay and Clair can quickly exceed PostgreSQL’s connection limits. Connection poolers such as pgBouncer are not supported with Project Quay or Clair. |
When managing your own PostgreSQL database for use with Project Quay, the following best practices are recommended:
-
pg_trgmextension: Thepg_trgmextension must be enabled on the database for a successful deployment. -
Backups: Perform regular database backups using PostgreSQL-native tools or your existing backup infrastructure. The Project Quay Operator does not manage database backups.
-
Restores: When restoring a backup, ensure that all Project Quay pods are stopped before beginning the restore process.
-
Storage sizing: When using the Operator-managed PostgreSQL database, the default storage allocation is 50 GiB. For external databases, you must ensure sufficient storage capacity for your environment, as the Operator does not handle volume resizing.
-
Monitoring: Monitor disk usage, connection limits, and query performance to prevent outages caused by resource exhaustion.
Configuring an external PostgreSQL connection
To integrate an existing PostgreSQL database with your Project Quay registry, you can set the postgres component to unmanaged and configure the DB_URI in the configBundleSecret. This lets you leverage your current database infrastructure instead of using the Operator-managed database.
|
Note
|
The following procedure uses the OpenShift Container Platform web console to configure the Project Quay registry to use an external PostgreSQL database. For most users, use the web console is simpler. This procedure can also be done by using the |
-
On the OpenShift Container Platform web console, click Operators → Installed Operators.
-
Click Red Hat Quay.
-
Click Quay Registry.
-
Click the name of your Project Quay registry, for example, example-registry.
-
Click YAML.
-
Set the
postgresfield of theQuayRegistryCR tomanaged: false. For example:- kind: postgres managed: false -
Click Save.
-
Click Details → the name of your
Config Bundle Secretresource. -
On the Secret Details page, click Actions → Edit Secret.
-
Add the
DB_URIfield to yourconfig.yamlfile. For example:DB_URI: postgresql://test-quay-database:postgres@test-quay-database:5432/test-quay-database -
Optional: Add additional database configuration fields, such as
DB_CONNECTION_ARGSor SSL/TLS connection arguments. For more information, see Database connection arguments. -
Click Save.
Configuring an external Redis connection
Manage your own Redis infrastructure by using an external Redis database with Project Quay.
To integrate an existing Redis database with your Project Quay registry, you can set the redis component to unmanaged and configure BUILDLOGS_REDIS and USER_EVENTS_REDIS in the configBundleSecret resource.
|
Important
|
Do not share the same Redis instance between Project Quay and Clair deployments. Each service must use its own dedicated Redis instance. Sharing Redis with other workloads is not supported, because connection-intensive components such as Project Quay and Clair can quickly exhaust available Redis connections and degrade performance. |
-
In the OpenShift Container Platform web console, navigate to Operators → Installed Operators.
-
Click Red Hat Quay.
-
Click QuayRegistry.
-
Click the name of your Project Quay registry, for example, example-registry.
-
Click YAML.
-
Set the
rediscomponent to unmanaged by adding the following entry underspec.components:- kind: redis managed: false -
Click Save.
-
Click Details → the name of your
Config Bundle Secretresource. -
On the Secret details page, click Actions → Edit Secret.
-
In the
config.yamlsection, add entries for your external Redis instance. For example:BUILDLOGS_REDIS: host: redis.example.com port: 6379 ssl: false USER_EVENTS_REDIS: host: redis.example.com port: 6379 ssl: falseImportantIf both the
BUILDLOGS_REDISandUSER_EVENTS_REDISfields reference the same Redis deployment, ensure that your Redis service can handle the combined connection load. For large or high-throughput registries, use separate Redis databases or clusters for these components. -
Optional: Add additional database configuration fields, such as
DB_CONNECTION_ARGSor SSL/TLS connection arguments. For more information, see Redis configuration fields. -
Click Save.
Configure AWS STS for object storage
Configure AWS Security Token Service (STS) for Amazon S3 object storage on standalone Red Hat Quay deployments.
Configuring Project Quay to use AWS STS
To configure Project Quay to use AWS STS for Amazon S3 storage, you can update the DISTRIBUTED_STORAGE_CONFIG block in your config.yaml file and restart the registry.
-
Update your
config.yamlfile for Project Quay to include the following information:# ... DISTRIBUTED_STORAGE_CONFIG: default: - STSS3Storage - sts_role_arn: <role_arn> s3_bucket: <s3_bucket_name> storage_path: <storage_path> s3_region: <region> sts_user_access_key: <s3_user_access_key> sts_user_secret_key: <s3_user_secret_key> # ...where:
sts_role_arn-
Specifies the unique Amazon Resource Name (ARN) required when configuring AWS STS.
s3_bucket-
Specifies the name of your S3 bucket.
storage_path-
Specifies the storage path for data. Usually
/datastorage. s3_region-
Specifies the Amazon Web Services region. Defaults to
us-east-1. sts_user_access_key-
Specifies the generated AWS S3 user access key required when configuring AWS STS.
sts_user_secret_key-
Specifies the generated AWS S3 user secret key required when configuring AWS STS.
-
Restart your Project Quay deployment.
-
Tag a sample image, for example,
busybox, that you push to the repository. For example:$ podman tag docker.io/library/busybox <quay-server.example.com>/<organization_name>/busybox:test -
Push the sample image by running the following command:
$ podman push <quay-server.example.com>/<organization_name>/busybox:test -
Verify that the push was successful by navigating to the Organization that you pushed the image to in your Project Quay registry → Tags.
-
Navigate to the Amazon Web Services (AWS) console and locate your S3 bucket.
-
Click the name of your S3 bucket.
-
On the Objects page, click datastorage/.
-
On the datastorage/ page, the following resources should appear:
-
sha256/
-
uploads/
These resources indicate that the push was successful, and that AWS STS is properly configured.
-
Configure networking
Configure IPv6 and dual-stack networking for standalone Project Quay deployments, and manage custom ingress, routes, and SSL/TLS for Operator deployments on OpenShift Container Platform.
IPv6 and dual-stack deployments
You can deploy standalone Project Quay on IPv6-only or dual-stack (IPv4 and IPv6) networks. Set FEATURE_LISTEN_IP_VERSION in your config.yaml file to enable the protocol family that your environment supports.
Some storage backends have known limitations on IPv6-only networks.
Enabling the IPv6 protocol family
To enable IPv6 support on your standalone Project Quay deployment, you can set FEATURE_LISTEN_IP_VERSION to IPv6 in your config.yaml file and restart the registry.
|
Warning
|
If your environment is configured for IPv4, but the |
-
Your host and container software platform (Docker, Podman) must be configured to support IPv6.
-
In your deployment’s
config.yamlfile, add theFEATURE_LISTEN_IP_VERSIONparameter and set it toIPv6, for example:FEATURE_GOOGLE_LOGIN: false FEATURE_INVITE_ONLY_USER_CREATION: false FEATURE_LISTEN_IP_VERSION: IPv6 FEATURE_MAILING: false FEATURE_NONSUPERUSER_TEAM_SYNCING_SETUP: false -
Start, or restart, your Project Quay deployment.
-
Check that your deployment is listening to IPv6 by entering the following command:
$ curl <quay_endpoint>/health/instanceExample output{"data":{"services":{"auth":true,"database":true,"disk_space":true,"registry_gunicorn":true,"service_key":true,"web_gunicorn":true}},"status_code":200}
-
After you enable IPv6 in your deployment’s
config.yamlfile, you can use all Project Quay features as usual when your environment is configured for IPv6 and is not affected by known IPv6 limitations.
Enabling the dual-stack protocol family
To enable dual-stack (IPv4 and IPv6) support on your standalone Project Quay deployment, you can set FEATURE_LISTEN_IP_VERSION to dual-stack in your config.yaml file and restart the registry.
-
Your host and container software platform (Docker, Podman) must be configured to support IPv6.
-
In your deployment’s
config.yamlfile, add theFEATURE_LISTEN_IP_VERSIONparameter and set it todual-stack, for example:FEATURE_GOOGLE_LOGIN: false FEATURE_INVITE_ONLY_USER_CREATION: false FEATURE_LISTEN_IP_VERSION: dual-stack FEATURE_MAILING: false FEATURE_NONSUPERUSER_TEAM_SYNCING_SETUP: false -
Start, or restart, your Project Quay deployment.
-
Check that your deployment is listening on both channels by entering the following commands:
-
For IPv4, enter the following command:
$ curl --ipv4 <quay_endpoint>Example output{"data":{"services":{"auth":true,"database":true,"disk_space":true,"registry_gunicorn":true,"service_key":true,"web_gunicorn":true}},"status_code":200} -
For IPv6, enter the following command:
$ curl --ipv6 <quay_endpoint>Example output{"data":{"services":{"auth":true,"database":true,"disk_space":true,"registry_gunicorn":true,"service_key":true,"web_gunicorn":true}},"status_code":200}
-
-
After you enable dual-stack in your deployment’s
config.yamlfile, you can use all Project Quay features as usual when your environment is configured for dual-stack.
IPv6 and dual-stack limitations
On IPv6 single-stack environments, Azure Blob Storage and Amazon S3 CloudFront endpoints that do not support IPv6 prevent those storage configurations from working with Project Quay.
-
Currently, attempting to configure your Project Quay deployment with the common Azure Blob Storage configuration does not work on IPv6 single-stack environments. Because the endpoint of Azure Blob Storage does not support IPv6, no workaround exists for this issue.
-
Currently, attempting to configure your Project Quay deployment with Amazon S3 CloudFront does not work on IPv6 single-stack environments. Because the endpoint of Amazon S3 CloudFront does not support IPv6, no workaround exists for this issue.
Configure custom ingress
You can configure custom ingress for Project Quay by disabling the Operator-managed route component and managing your own routes or ingress controllers.
Managing your own routes or ingress controllers is useful when your environment requires a custom SSL/TLS setup, specific DNS naming conventions, or when Project Quay is deployed behind a load balancer or proxy that handles TLS termination.
The Project Quay Operator separates route management from SSL/TLS configuration by introducing a distinct tls component. You can therefore manage each independently, depending on whether Project Quay or the cluster should handle TLS termination. For more information about using SSL/TLS certificates with your deployment, see "Securing Project Quay".
|
Note
|
If you disable the managed |
Disabling the Route component
To prevent the Project Quay Operator from creating a route, you can set the route component to unmanaged in the QuayRegistry custom resource. You must then configure SSL/TLS handling in your config.yaml file.
-
In your
quayregistry.yamlfile, set theroutecomponent asmanaged: false:apiVersion: quay.redhat.com/v1 kind: QuayRegistry metadata: name: example-registry namespace: quay-enterprise spec: components: - kind: route managed: false -
In your
config.yamlfile, configure Project Quay to handle SSL/TLS. For example:# ... EXTERNAL_TLS_TERMINATION: false SERVER_HOSTNAME: example-registry-quay-quay-enterprise.apps.user1.example.com PREFERRED_URL_SCHEME: https # ...If the configuration is incomplete, the following error might appear:
{ "reason":"ConfigInvalid", "message":"required component `route` marked as unmanaged, but `configBundleSecret` is missing necessary fields" }
Configure SSL/TLS and routes
Configuring SSL/TLS and routes for Project Quay lets you control how TLS termination and route management work together. The tls component provides support for OpenShift Container Platform edge termination routes and enables independent control of route management and TLS certificate handling.
EXTERNAL_TLS_TERMINATION: true is the default, opinionated setting, which assumes the cluster manages TLS termination.
|
Note
|
|
Multiple valid configurations are possible, as shown in the following table:
| Option | Route | TLS | Certs provided | Result |
|---|---|---|---|---|
My own load balancer handles TLS |
Managed |
Managed |
No |
Edge route using default cluster wildcard certificate |
Project Quay handles TLS |
Managed |
Unmanaged |
Yes |
Passthrough route with certificates mounted in the Project Quay pod |
Project Quay handles TLS |
Unmanaged |
Unmanaged |
Yes |
Certificates set inside the Project Quay pod; user must manually create a route |
Configure Elasticsearch and Splunk action log storage
Configure action log storage in Elasticsearch or Splunk, including Splunk installation prerequisites, bearer token generation, Project Quay configuration, and log verification.
Action log storage backends (Elasticsearch and Splunk)
By default, usage logs are stored in the Project Quay database and exposed through the web UI on organization and repository levels.
Appropriate administrative privileges are required to see log entries. For deployments with a large amount of logged operations, you can store the usage logs in Elasticsearch and Splunk instead of the Project Quay database backend.
Configure action log storage for Elasticsearch
To store Project Quay action logs in Elasticsearch, you can update the LOGS_MODEL settings in your config.yaml file and restart the registry. Usage logs remain available in the web UI for repositories and organizations.
|
Note
|
To configure action log storage for Elasticsearch, you must provide your own Elasticsearch stack; Project Quay does not include Elasticsearch as a customizable component. |
-
Obtain an Elasticsearch account.
-
Update your Project Quay
config.yamlfile to include the following information:# ... LOGS_MODEL: elasticsearch LOGS_MODEL_CONFIG: producer: elasticsearch elasticsearch_config: host: http://<host.elasticsearch.example>:<port> port: 9200 access_key: <access_key> secret_key: <secret_key> use_ssl: True index_prefix: <logentry> aws_region: <us-east-1> # ...where:
LOGS_MODEL-
Specifies the method for handling log data.
LOGS_MODEL_CONFIG.producer-
Specifies either Elasticsearch or Kinesis to direct logs to an intermediate Kinesis stream on AWS. You need to configure your own pipeline to send logs from Kinesis to Elasticsearch, for example, Logstash.
LOGS_MODEL_CONFIG.elasticsearch_config.host-
Specifies the hostname or IP address of the system providing the Elasticsearch service.
LOGS_MODEL_CONFIG.elasticsearch_config.port-
Specifies the port number providing the Elasticsearch service on the host you just entered. Note that the port must be accessible from all systems running the Project Quay registry. The default is TCP port
9200. LOGS_MODEL_CONFIG.elasticsearch_config.access_key-
Specifies the access key needed to gain access to the Elasticsearch service, if required.
LOGS_MODEL_CONFIG.elasticsearch_config.secret_key-
Specifies the secret key needed to gain access to the Elasticsearch service, if required.
LOGS_MODEL_CONFIG.elasticsearch_config.use_ssl-
Specifies whether to use SSL/TLS for Elasticsearch. Defaults to
True. LOGS_MODEL_CONFIG.elasticsearch_config.index_prefix-
Specifies a prefix to attach to log entries.
LOGS_MODEL_CONFIG.elasticsearch_config.aws_region-
Specifies the AWS region if you are running on AWS. Otherwise, leave it blank.
-
Optional. If you are using Kinesis as your logs producer, you must include the following fields in your
config.yamlfile:kinesis_stream_config: stream_name: <kinesis_stream_name> access_key: <aws_access_key> secret_key: <aws_secret_key> aws_region: <aws_region>where:
kinesis_stream_config.stream_name-
Specifies the name of the Kinesis stream.
kinesis_stream_config.access_key-
Specifies the name of the AWS access key needed to gain access to the Kinesis stream, if required.
kinesis_stream_config.secret_key-
Specifies the name of the AWS secret key needed to gain access to the Kinesis stream, if required.
kinesis_stream_config.aws_region-
Specifies the Amazon Web Services (AWS) region.
-
Save your
config.yamlfile and restart your Project Quay deployment.
Configure action log storage for Splunk
Splunk is an alternative to Elasticsearch for storing and analyzing Project Quay action logs. You can forward logs directly to Splunk or to the Splunk HTTP Event Collector (HEC) during or after deployment.
Installing and creating a username for Splunk
To prepare Splunk for Project Quay action log storage, you can install Splunk Enterprise and create an administrator username and password.
-
Create a Splunk account by navigating to Splunk and entering the required credentials.
-
Navigate to the Splunk Enterprise Free Trial page, select your platform and installation package, and then click Download Now.
-
Install the Splunk software on your machine. When prompted, create a username, for example,
splunk_adminand password. -
After creating a username and password, a localhost URL will be provided for your Splunk deployment, for example,
http://<sample_url>.remote.csb:8000/. Open the URL in your preferred browser. -
Log in with the username and password you created during installation. You are directed to the Splunk UI.
Generating a Splunk bearer token
You can generate a Splunk bearer token for Project Quay action log forwarding by using the Splunk UI or the CLI.
Generating a Splunk bearer token using the Splunk UI
To create a Splunk bearer token for Project Quay from the Splunk UI, you can enable token authentication and create a new token.
-
You have installed Splunk and created a username.
-
On the Splunk UI, navigate to Settings → Tokens.
-
Click Enable Token Authentication.
-
Ensure that Token Authentication is enabled by clicking Token Settings and selecting Token Authentication if necessary.
-
Optional: Set the expiration time for your token. This defaults at 30 days.
-
Click Save.
-
Click New Token.
-
Enter information for User and Audience.
-
Optional: Set the Expiration and Not Before information.
-
Click Create. Your token appears in the Token box. Copy the token immediately.
ImportantIf you close out of the box before copying the token, you must create a new token. The token in its entirety is not available after closing the New Token window.
Generating a Splunk bearer token using the CLI
To create a Splunk bearer token for Project Quay from the CLI, you can enable token authentication and request a token with curl.
-
You have installed Splunk and created a username.
-
In your CLI, enter the following
CURLcommand to enable token authentication, passing in your Splunk username and password:$ curl -k -u <username>:<password> -X POST <scheme>://<host>:<port>/services/admin/token-auth/tokens_auth -d disabled=false -
Create a token by entering the following
CURLcommand, passing in your Splunk username and password.$ curl -k -u <username>:<password> -X POST <scheme>://<host>:<port>/services/authorization/tokens?output_mode=json --data name=<username> --data audience=Users --data-urlencode expires_on=+30d -
Save the generated bearer token.
Generating an HEC ingest token
To forward Project Quay action logs to Splunk through the HTTP Event Collector (HEC), you can generate an HEC ingest token in the Splunk web UI or by using the Splunk REST API.
|
Note
|
Splunk HEC tokens are ingest-only and cannot search. |
-
You have installed Splunk and created a username.
-
To create an HEC token using the Splunk web UI:
-
Log in to the Splunk via the web UI.
-
Click Settings → Data Inputs → HTTP Event Collector.
-
Click New Token.
-
Name the token, for example,
quay-hec, and select the target index, for example,quay_logs. -
Click Submit and copy the token value.
-
-
To create an HEC token using the Splunk REST API:
-
Enable HEC by entering the following command:
$ curl -k -u <username>:<password> \ https://<splunk.example.com>:8089/servicesNS/admin/splunk_httpinput/data/inputs/http/http \ -d "disabled=0" -
Create an HEC token by entering the following command:
$ curl -k -u <username>:<password> \ "https://<splunk.example.com>:8089/servicesNS/admin/splunk_httpinput/data/inputs/http?output_mode=json" \ -d "name=quay-hec" -d "index=quay_logs"Example output:{"entry":[{"content":{"token":"<your_bearer_token>"}}]}
-
Configuring Project Quay to use Splunk
To send Project Quay action logs to Splunk or the Splunk HTTP Event Collector (HEC), you can add the Splunk settings to your config.yaml file and restart the registry.
-
You have installed Splunk and created a username.
-
You have generated a Splunk bearer token.
-
Configure Project Quay to use Splunk or the Splunk HTTP Event Collector (HEC).
-
If opting to use Splunk, open your Project Quay
config.yamlfile and add the following configuration fields:# ... LOGS_MODEL: splunk LOGS_MODEL_CONFIG: producer: splunk splunk_config: host: http://<user_name>.remote.csb port: 8089 bearer_token: <bearer_token> url_scheme: <http/https> verify_ssl: False index_prefix: <splunk_log_index_name> ssl_ca_path: <location_to_ssl-ca-cert.pem> search_timeout: 60 max_results: 10000 export_batch_size: 5000 # ...where:
LOGS_MODEL_CONFIG.splunk_config.host-
Specifies the Splunk cluster endpoint.
LOGS_MODEL_CONFIG.splunk_config.port-
Specifies the Splunk management cluster endpoint port. Differs from the Splunk GUI hosted port. Can be found on the Splunk UI under Settings → Server Settings → General Settings.
LOGS_MODEL_CONFIG.splunk_config.bearer_token-
Specifies the generated bearer token for Splunk.
LOGS_MODEL_CONFIG.splunk_config.url_scheme-
Specifies the URL scheme for access the Splunk service. If Splunk is configured to use TLS/SSL, this must be
https. LOGS_MODEL_CONFIG.splunk_config.verify_ssl-
Specifies whether to enable TLS/SSL. Defaults to
True. LOGS_MODEL_CONFIG.splunk_config.index_prefix-
Specifies the Splunk index prefix. Can be a new, or used, index. Can be created from the Splunk UI.
LOGS_MODEL_CONFIG.splunk_config.ssl_ca_path-
Specifies the relative container path to a single
.pemfile containing a certificate authority (CA) for TLS/SSL validation. LOGS_MODEL_CONFIG.splunk_config.search_timeout-
Specifies the timeout for Splunk search queries in seconds. Increase for slow Splunk clusters or complex queries.
LOGS_MODEL_CONFIG.splunk_config.max_results-
Specifies the maximum number of results to return per search query. Larger values require more memory.
LOGS_MODEL_CONFIG.splunk_config.export_batch_size-
Specifies the batch size for log export operations.
-
If opting to use Splunk HEC, open your Project Quay
config.yamlfile and add the following configuration fields:# ... LOGS_MODEL: splunk LOGS_MODEL_CONFIG: producer: splunk_hec splunk_hec_config: host: prd-p-aaaaaq.splunkcloud.com port: 8088 hec_token: 12345678-1234-1234-1234-1234567890ab url_scheme: https verify_ssl: False index: quay splunk_host: quay-dev splunk_sourcetype: quay_logs timeout: 10 search_token: <bearer_token> search_host: <splunk.example.com> search_port: 8089 search_timeout: 60 max_results: 10000 export_batch_size: 5000 # ...where:
LOGS_MODEL_CONFIG.producer-
Specifies
splunk_hecwhen configuring Splunk HEC. LOGS_MODEL_CONFIG.splunk_hec_config-
Specifies the logs model configuration for Splunk HTTP Event Collector action logs configuration.
LOGS_MODEL_CONFIG.splunk_hec_config.host-
Specifies the Splunk cluster endpoint.
LOGS_MODEL_CONFIG.splunk_hec_config.port-
Specifies the Splunk management cluster endpoint port.
LOGS_MODEL_CONFIG.splunk_hec_config.hec_token-
Specifies the HEC token for Splunk.
LOGS_MODEL_CONFIG.splunk_hec_config.url_scheme-
Specifies the URL scheme for access the Splunk service. If Splunk is behind SSL/TLS, must be
https. LOGS_MODEL_CONFIG.splunk_hec_config.verify_ssl-
Specifies whether to enable (
true) or disable (false) SSL/TLS verification for HTTPS connections. LOGS_MODEL_CONFIG.splunk_hec_config.index-
Specifies the Splunk index to use.
LOGS_MODEL_CONFIG.splunk_hec_config.splunk_host-
Specifies the host name to log this event.
LOGS_MODEL_CONFIG.splunk_hec_config.splunk_sourcetype-
Specifies the name of the Splunk
sourcetypeto use. LOGS_MODEL_CONFIG.splunk_hec_config.timeout-
Specifies the timeout in seconds for HTTP requests to the Splunk HEC endpoint. Prevents requests from hanging indefinitely when Splunk is unresponsive.
LOGS_MODEL_CONFIG.splunk_hec_config.search_token-
Specifies an optional bearer token for the Splunk search API. Required because HEC tokens are ingest-only and cannot search.
LOGS_MODEL_CONFIG.splunk_hec_config.search_host-
Specifies the Splunk management host for the search API. Defaults to the HEC host if not specified.
LOGS_MODEL_CONFIG.splunk_hec_config.search_port-
Specifies the Splunk management port for the search API. Defaults to
8089if not specified. LOGS_MODEL_CONFIG.splunk_hec_config.search_timeout-
Specifies the timeout for Splunk search queries in seconds. Increase for slow Splunk clusters or complex queries.
LOGS_MODEL_CONFIG.splunk_hec_config.max_results-
Specifies the maximum number of results to return per search query. Larger values require more memory.
LOGS_MODEL_CONFIG.splunk_hec_config.export_batch_size-
Specifies the batch size for log export operations.
-
-
If you are configuring
ssl_ca_path, you must configure the SSL/TLS certificate so that Project Quay trusts it.-
If you are using a standalone deployment of Project Quay, SSL/TLS certificates can be provided by placing the certificate file inside of the
extra_ca_certsdirectory, or inside of the relative container path and specified byssl_ca_path. -
If you are using the Project Quay Operator, create a config bundle secret, including the certificate authority (CA) of the Splunk server. For example:
$ oc create secret generic --from-file config.yaml=./config_390.yaml --from-file extra_ca_cert_splunkserver.crt=./splunkserver.crt config-bundle-secretSpecify the
conf/stack/extra_ca_certs/splunkserver.crtfile in yourconfig.yaml. For example:# ... LOGS_MODEL: splunk LOGS_MODEL_CONFIG: producer: splunk splunk_config: host: ec2-12-345-67-891.us-east-2.compute.amazonaws.com port: 8089 bearer_token: eyJra url_scheme: https verify_ssl: true index_prefix: quay123456 ssl_ca_path: conf/stack/splunkserver.crt # ...
-
Creating an action log
To verify that Project Quay is forwarding action logs to Splunk, you can create a robot account in an organization and search the Splunk index for the forwarded JSON log entries.
-
You have installed Splunk and created a username.
-
You have generated a Splunk bearer token.
-
You have configured your Project Quay
config.yamlfile to enable Splunk.
-
Log in to your Project Quay deployment.
-
Click on the name of the organization that you use to create an action log for Splunk.
-
In the navigation pane, click Robot Accounts → Create Robot Account.
-
When prompted, enter a name for the robot account, for example
splunkrobotaccount, then click Create robot account. -
On your browser, open the Splunk UI.
-
Click Search and Reporting.
-
In the search bar, enter the name of your index, for example,
<splunk_log_index_name>and press Enter.The search results populate on the Splunk UI. Logs are forwarded in JSON format. A response might look similar to the following:
{ "log_data": { "kind": "authentication", "account": "quayuser123", "performer": "John Doe", "repository": "projectQuay", "ip": "192.168.1.100", "metadata_json": {...}, "datetime": "2024-02-06T12:30:45Z" } }where:
kind-
Specifies the type of log event. In this example,
authenticationindicates that the log entry relates to an authentication event. account-
Specifies the user account involved in the event.
performer-
Specifies the individual who performed the action.
repository-
Specifies the repository associated with the event.
ip-
Specifies the IP address from which the action was performed.
metadata_json-
Specifies additional metadata related to the event, when present.
datetime-
Specifies the timestamp of when the event occurred.
Configure notifications for registry events
Configure repository event notifications by using the web UI or API, set up quota and image expiration alerts, route email through organization contact addresses, and review repository event payload formats.
Notifications overview
Notifications in Project Quay alert you about repository events such as pushes, builds, and image expiry. You can configure delivery methods for users, teams, or organizations.
Notification entries appear on the repository Events and Notifications page and in the global Notifications panel. Delivery methods include email, webhook POST, Flowdock, HipChat, and Slack.
Creating notifications by using the UI
To alert users to repository events such as pushes or build failures, you can create a notification from the Settings page in the Project Quay v2 UI. You select an event trigger and a delivery method, such as email or Slack.
-
You have created a repository.
-
You have administrative privileges for the repository.
-
In the navigation pane, click Settings.
-
In the Events and Notifications category, click Create Notification to add a new notification for a repository event. The Create notification popup box appears.
-
On the Create repository popup box, click the When this event occurs box to select an event. You can select a notification for the following types of events:
-
Push to Repository
-
Image build failed
-
Image build queued
-
Image build started
-
Image build success
-
Image build cancelled
-
Image expiry trigger
-
-
After you have selected the event type, select the notification method. The following methods are supported:
-
Quay Notification
-
E-mail Notification
-
Webhook POST
-
Flowdock Team Notification
-
HipChat Room Notification
-
Slack Notification
Depending on the method that you choose, you must include additional information. For example, if you select E-mail, you are required to include an e-mail address and an optional notification title.
-
-
After selecting an event and notification method, click Create Notification.
Creating an image expiration notification
To create an image expiration notification in Project Quay, you can use the v2 UI or the API. You set how many days before expiry to send the alert.
Triggers can work in conjunction with the auto-pruning feature. You can also create this notification by using the createRepoNotification API endpoint.
-
FEATURE_GARBAGE_COLLECTION: trueis set in yourconfig.yamlfile. -
Optional.
FEATURE_AUTO_PRUNE: trueis set in yourconfig.yamlfile.
-
On the Project Quay v2 UI, click Repositories.
-
Select the name of a repository.
-
Click Settings → Events and notifications.
-
Click Create notification. The Create notification popup box appears.
-
Click the Select event… box, then click Image expiry trigger.
-
In the When the image is due to expiry in days box, enter the number of days before the image’s expiration when you want to receive an alert. For example, use
1for 1 day. -
In the Select method… box, click one of the following:
-
E-mail
-
Webhook POST
-
Flowdock Team Notification
-
HipChat Room Notification
-
Slack Notification
-
-
Depending on which method you chose, include the necessary data. For example, if you chose Webhook POST, include the
Webhook URL. -
Optional. Provide a POST JSON body template.
-
Optional. Provide a Title for your notification.
-
Click Submit. You are returned to the Events and notifications page, and the notification now appears.
-
Optional. You can set the
NOTIFICATION_TASK_RUN_MINIMUM_INTERVAL_MINUTESvariable in your config.yaml file. with this field set, if there are any expiring images notifications will be sent automatically. By default, this is set to300, or 5 hours, however it can be adjusted as warranted.NOTIFICATION_TASK_RUN_MINIMUM_INTERVAL_MINUTES: 300where:
NOTIFICATION_TASK_RUN_MINIMUM_INTERVAL_MINUTES-
Specifies that by default, this field is set to
300, or 5 hours.
-
Click the menu kebab → Test Notification. The following message is returned:
Test Notification Queued A test version of this notification has been queued and should appear shortly -
Depending on which method you chose, check your e-mail, webhook address, Slack channel, and so on. The information sent should look similar to the following example:
{ "repository": "sample_org/busybox", "namespace": "sample_org", "name": "busybox", "docker_url": "quay-server.example.com/sample_org/busybox", "homepage": "http://quay-server.example.com/repository/sample_org/busybox", "tags": [ "latest", "v1" ], "expiring_in": "1 days" }
Creating notifications by using the API
To create, test, reset, or delete repository notifications in Project Quay, you can use the repository notification API endpoints.
-
You have created a repository.
-
You have administrative privileges for the repository.
-
You have created an OAuth access token.
-
Enter the following
POST /api/v1/repository/{repository}/notificationcommand to create a notification on your repository:$ curl -X POST \ -H "Authorization: Bearer <bearer_token>" \ -H "Content-Type: application/json" \ --data '{ "event": "<event>", "method": "<method>", "config": { "<config_key>": "<config_value>" }, "eventConfig": { "<eventConfig_key>": "<eventConfig_value>" } }' \ https://<quay-server.example.com>/api/v1/repository/<namespace>/<repository_name>/notification/This command does not return output in the CLI. Instead, you can enter the following
GET /api/v1/repository/{repository}/notification/{uuid}command to obtain information about the repository notification:{"uuid": "240662ea-597b-499d-98bb-2b57e73408d6", "title": null, "event": "repo_push", "method": "quay_notification", "config": {"target": {"name": "quayadmin", "kind": "user", "is_robot": false, "avatar": {"name": "quayadmin", "hash": "b28d563a6dc76b4431fc7b0524bbff6b810387dac86d9303874871839859c7cc", "color": "#17becf", "kind": "user"}}}, "event_config": {}, "number_of_failures": 0} -
You can test your repository notification by entering the following
POST /api/v1/repository/{repository}/notification/{uuid}/testcommand:$ curl -X POST \ -H "Authorization: Bearer <bearer_token>" \ https://<quay-server.example.com>/api/v1/repository/<repository>/notification/<uuid>/testExample output{} -
You can reset repository notification failures to 0 by entering the following
POST /api/v1/repository/{repository}/notification/{uuid}command:$ curl -X POST \ -H "Authorization: Bearer <bearer_token>" \ https://<quay-server.example.com>/api/v1/repository/<repository>/notification/<uuid> -
Enter the following
DELETE /api/v1/repository/{repository}/notification/{uuid}command to delete a repository notification:$ curl -X DELETE \ -H "Authorization: Bearer <bearer_token>" \ https://<quay-server.example.com>/api/v1/repository/<namespace>/<repository_name>/notification/<uuid>This command does not return output in the CLI. Instead, you can enter the following
GET /api/v1/repository/{repository}/notification/command to retrieve a list of all notifications:$ curl -X GET -H "Authorization: Bearer <bearer_token>" -H "Accept: application/json" https://<quay-server.example.com>/api/v1/repository/<namespace>/<repository_name>/notificationExample output{"notifications": []}
Creating quota notifications by using the API
To create quota warning or error notifications for organization and user namespaces in Project Quay, you can use the API. Warning limits trigger quota_warning events and reject limits trigger quota_error events.
Warning-type quota limits trigger quota_warning notifications. Reject-type quota limits trigger quota_error notifications. For organization email notifications, "config": {} is valid because recipients are resolved server-side (organization contact email, or organization administrators if unset). User email notifications are sent to the user account email address.
Project Quay throttles repeated alerts for the same namespace and threshold by using QUOTA_NOTIFICATION_COOLDOWN_SECONDS (default: 86400 / 24 hours). If usage drops below a threshold and later crosses it again, the notification can re-fire. Deleting a quota also removes associated quota_warning and quota_error notification rules for that namespace.
-
You have created an OAuth access token.
-
FEATURE_QUOTA_MANAGEMENTis enabled in your Project Quay configuration. -
FEATURE_QUOTA_NOTIFICATIONSis enabled in your Project Quay configuration. -
Optional: You have established quota limits for your organization or user namespace. You can create notification rules before setting quota limits. Notifications trigger automatically after quotas are configured and thresholds are crossed.
-
You have administrative privileges for the namespace.
-
To create an organization quota notification, enter a command similar to the following example:
$ curl -X POST \ -H "Authorization: Bearer <bearer_token>" \ -H "Content-Type: application/json" \ --data '{ "event": "quota_warning", "method": "email", "config": {}, "eventConfig": {} }' \ https://<quay-server.example.com>/api/v1/organization/<orgname>/notificationsExample output{ "uuid": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "title": null, "event": "quota_warning", "method": "email", "config": {}, "event_config": {}, "number_of_failures": 0 } -
Optional: You can create a quota error notification by changing the event type. To complete this task, run a command similar to the following example:
$ curl -X POST \ -H "Authorization: Bearer <bearer_token>" \ -H "Content-Type: application/json" \ --data '{ "event": "quota_error", "method": "slack", "config": { "url": "https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK" }, "eventConfig": {} }' \ https://<quay-server.example.com>/api/v1/organization/<orgname>/notifications -
To create a user namespace quota notification, enter a command similar to the following example:
$ curl -X POST \ -H "Authorization: Bearer <bearer_token>" \ -H "Content-Type: application/json" \ --data '{ "event": "quota_warning", "method": "email", "config": {}, "eventConfig": {} }' \ https://<quay-server.example.com>/api/v1/user/namespacenotifications
Configuring email routing with organization contact email
To route quota notification emails to a shared organization contact address in Project Quay, you can set the organization contact email with the API. You can list and test notifications for organizations and user namespaces.
-
Set the organization contact email by entering a command similar to the following example:
$ curl -X PUT \ -H "Authorization: Bearer <bearer_token>" \ -H "Content-Type: application/json" \ --data '{ "email": "ops-team@example.com" }' \ https://<quay-server.example.com>/api/v1/organization/<orgname>NoteSetting the organization contact email requires organization administrator permissions (
org:adminscope). If a contact email is not set, quota notifications default to sending to organization administrator email addresses. -
List all notifications for an organization by entering a command similar to the following example:
$ curl -X GET \ -H "Authorization: Bearer <bearer_token>" \ -H "Accept: application/json" \ https://<quay-server.example.com>/api/v1/organization/<orgname>/notificationsExample output{ "notifications": [ { "uuid": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "title": null, "event": "quota_warning", "method": "email", "config": {}, "event_config": {}, "number_of_failures": 0 } ] } -
List all notifications for your user namespace by entering a command similar to the following example:
$ curl -X GET \ -H "Authorization: Bearer <bearer_token>" \ -H "Accept: application/json" \ https://<quay-server.example.com>/api/v1/user/namespacenotifications -
To test a notification, enter a command similar to the following example:
$ curl -X POST \ -H "Authorization: Bearer <bearer_token>" \ https://<quay-server.example.com>/api/v1/organization/<orgname>/notifications/<uuid>/testExample output{}NoteFor email notifications, if an organization contact email is set, the test notification is routed to that address. Otherwise, email notifications default to organization administrators. For Slack, webhook, and other methods, routing follows the notification
config.
Repository events description
Repository events in Project Quay describe the triggers that can generate notifications. You can use these event types when you configure alerts.
Repository Push
A successful push of one or more images was made to the repository:
{
"name": "repository",
"repository": "dgangaia/test",
"namespace": "dgangaia",
"docker_url": "quay.io/dgangaia/test",
"homepage": "https://quay.io/repository/dgangaia/repository",
"updated_tags": [
"latest"
]
}
Dockerfile Build Queued
The following example is a response from a Dockerfile Build that has been queued into the Build system.
|
Note
|
Responses can differ based on the use of optional attributes. |
{
"build_id": "296ec063-5f86-4706-a469-f0a400bf9df2",
"trigger_kind": "github", //Optional
"name": "test",
"repository": "dgangaia/test",
"namespace": "dgangaia",
"docker_url": "quay.io/dgangaia/test",
"trigger_id": "38b6e180-9521-4ff7-9844-acf371340b9e", //Optional
"docker_tags": [
"master",
"latest"
],
"repo": "test",
"trigger_metadata": {
"default_branch": "master",
"commit": "b7f7d2b948aacbe844ee465122a85a9368b2b735",
"ref": "refs/heads/master",
"git_url": "git@github.com:dgangaia/test.git",
"commit_info": { //Optional
"url": "https://github.com/dgangaia/test/commit/b7f7d2b948aacbe844ee465122a85a9368b2b735",
"date": "2019-03-06T12:48:24+11:00",
"message": "adding 5",
"author": { //Optional
"username": "dgangaia",
"url": "https://github.com/dgangaia", //Optional
"avatar_url": "https://avatars1.githubusercontent.com/u/43594254?v=4" //Optional
},
"committer": {
"username": "web-flow",
"url": "https://github.com/web-flow",
"avatar_url": "https://avatars3.githubusercontent.com/u/19864447?v=4"
}
}
},
"is_manual": false,
"manual_user": null,
"homepage": "https://quay.io/repository/dgangaia/test/build/296ec063-5f86-4706-a469-f0a400bf9df2"
}
Dockerfile Build started
The following example is a response from a Dockerfile Build that has been queued into the Build system.
|
Note
|
Responses can differ based on the use of optional attributes. |
{
"build_id": "a8cc247a-a662-4fee-8dcb-7d7e822b71ba",
"trigger_kind": "github", //Optional
"name": "test",
"repository": "dgangaia/test",
"namespace": "dgangaia",
"docker_url": "quay.io/dgangaia/test",
"trigger_id": "38b6e180-9521-4ff7-9844-acf371340b9e", //Optional
"docker_tags": [
"master",
"latest"
],
"build_name": "50bc599",
"trigger_metadata": { //Optional
"commit": "50bc5996d4587fd4b2d8edc4af652d4cec293c42",
"ref": "refs/heads/master",
"default_branch": "master",
"git_url": "git@github.com:dgangaia/test.git",
"commit_info": { //Optional
"url": "https://github.com/dgangaia/test/commit/50bc5996d4587fd4b2d8edc4af652d4cec293c42",
"date": "2019-03-06T14:10:14+11:00",
"message": "test build",
"committer": { //Optional
"username": "web-flow",
"url": "https://github.com/web-flow", //Optional
"avatar_url": "https://avatars3.githubusercontent.com/u/19864447?v=4" //Optional
},
"author": { //Optional
"username": "dgangaia",
"url": "https://github.com/dgangaia", //Optional
"avatar_url": "https://avatars1.githubusercontent.com/u/43594254?v=4" //Optional
}
}
},
"homepage": "https://quay.io/repository/dgangaia/test/build/a8cc247a-a662-4fee-8dcb-7d7e822b71ba"
}
Dockerfile Build successfully completed
The following example is a response from a Dockerfile Build that has been successfully completed by the Build system.
|
Note
|
This event occurs simultaneously with a Repository Push event for the built image or images. |
{
"build_id": "296ec063-5f86-4706-a469-f0a400bf9df2",
"trigger_kind": "github", //Optional
"name": "test",
"repository": "dgangaia/test",
"namespace": "dgangaia",
"docker_url": "quay.io/dgangaia/test",
"trigger_id": "38b6e180-9521-4ff7-9844-acf371340b9e", //Optional
"docker_tags": [
"master",
"latest"
],
"build_name": "b7f7d2b",
"image_id": "sha256:0339f178f26ae24930e9ad32751d6839015109eabdf1c25b3b0f2abf8934f6cb",
"trigger_metadata": {
"commit": "b7f7d2b948aacbe844ee465122a85a9368b2b735",
"ref": "refs/heads/master",
"default_branch": "master",
"git_url": "git@github.com:dgangaia/test.git",
"commit_info": { //Optional
"url": "https://github.com/dgangaia/test/commit/b7f7d2b948aacbe844ee465122a85a9368b2b735",
"date": "2019-03-06T12:48:24+11:00",
"message": "adding 5",
"committer": { //Optional
"username": "web-flow",
"url": "https://github.com/web-flow", //Optional
"avatar_url": "https://avatars3.githubusercontent.com/u/19864447?v=4" //Optional
},
"author": { //Optional
"username": "dgangaia",
"url": "https://github.com/dgangaia", //Optional
"avatar_url": "https://avatars1.githubusercontent.com/u/43594254?v=4" //Optional
}
}
},
"homepage": "https://quay.io/repository/dgangaia/test/build/296ec063-5f86-4706-a469-f0a400bf9df2",
"manifest_digests": [
"quay.io/dgangaia/test@sha256:2a7af5265344cc3704d5d47c4604b1efcbd227a7a6a6ff73d6e4e08a27fd7d99",
"quay.io/dgangaia/test@sha256:569e7db1a867069835e8e97d50c96eccafde65f08ea3e0d5debaf16e2545d9d1"
]
}
Dockerfile Build failed
The following example is a response from a Dockerfile Build that has failed.
{
"build_id": "5346a21d-3434-4764-85be-5be1296f293c",
"trigger_kind": "github", //Optional
"name": "test",
"repository": "dgangaia/test",
"docker_url": "quay.io/dgangaia/test",
"error_message": "Could not find or parse Dockerfile: unknown instruction: GIT",
"namespace": "dgangaia",
"trigger_id": "38b6e180-9521-4ff7-9844-acf371340b9e", //Optional
"docker_tags": [
"master",
"latest"
],
"build_name": "6ae9a86",
"trigger_metadata": { //Optional
"commit": "6ae9a86930fc73dd07b02e4c5bf63ee60be180ad",
"ref": "refs/heads/master",
"default_branch": "master",
"git_url": "git@github.com:dgangaia/test.git",
"commit_info": { //Optional
"url": "https://github.com/dgangaia/test/commit/6ae9a86930fc73dd07b02e4c5bf63ee60be180ad",
"date": "2019-03-06T14:18:16+11:00",
"message": "failed build test",
"committer": { //Optional
"username": "web-flow",
"url": "https://github.com/web-flow", //Optional
"avatar_url": "https://avatars3.githubusercontent.com/u/19864447?v=4" //Optional
},
"author": { //Optional
"username": "dgangaia",
"url": "https://github.com/dgangaia", //Optional
"avatar_url": "https://avatars1.githubusercontent.com/u/43594254?v=4" //Optional
}
}
},
"homepage": "https://quay.io/repository/dgangaia/test/build/5346a21d-3434-4764-85be-5be1296f293c"
}
Dockerfile Build cancelled
The following example is a response from a Dockerfile Build that has been cancelled.
{
"build_id": "cbd534c5-f1c0-4816-b4e3-55446b851e70",
"trigger_kind": "github",
"name": "test",
"repository": "dgangaia/test",
"namespace": "dgangaia",
"docker_url": "quay.io/dgangaia/test",
"trigger_id": "38b6e180-9521-4ff7-9844-acf371340b9e",
"docker_tags": [
"master",
"latest"
],
"build_name": "cbce83c",
"trigger_metadata": {
"commit": "cbce83c04bfb59734fc42a83aab738704ba7ec41",
"ref": "refs/heads/master",
"default_branch": "master",
"git_url": "git@github.com:dgangaia/test.git",
"commit_info": {
"url": "https://github.com/dgangaia/test/commit/cbce83c04bfb59734fc42a83aab738704ba7ec41",
"date": "2019-03-06T14:27:53+11:00",
"message": "testing cancel build",
"committer": {
"username": "web-flow",
"url": "https://github.com/web-flow",
"avatar_url": "https://avatars3.githubusercontent.com/u/19864447?v=4"
},
"author": {
"username": "dgangaia",
"url": "https://github.com/dgangaia",
"avatar_url": "https://avatars1.githubusercontent.com/u/43594254?v=4"
}
}
},
"homepage": "https://quay.io/repository/dgangaia/test/build/cbd534c5-f1c0-4816-b4e3-55446b851e70"
}
Configure build worker environments
Configure TLS, bare metal, and virtual build worker environments for Project Quay on OpenShift Container Platform, including managed-route constraints, optional EC2 builder fallback, and object storage prerequisites for virtual builds.
Configuring the OpenShift Container Platform TLS component for builds
To enable the builds feature with unmanaged TLS in Project Quay, you can add the builder route name to the Subject Alternative Name (SAN) field in your SSL/TLS certificate configuration.
The tls component of the QuayRegistry custom resource definition (CRD) allows you to control whether SSL/TLS are managed by the Project Quay Operator, or self managed. In its current state, Project Quay does not support the builds feature, or the builder workers, when the tls component is managed by the Project Quay Operator.
When setting the tls component to unmanaged, you must supply your own ssl.cert and ssl.key files. Additionally, if you want your cluster to support builders, or the worker nodes that are responsible for building images, you must add both the Quay route and the builder route name to the SAN list in the certificate. Alternatively, however, you could use a wildcard.
-
You have set the
tlscomponent tounmanagedand uploaded custom SSL/TLS certificates to the Project Quay Operator. For more information, see SSL and TLS for Project Quay.
-
In the configuration file that defines your SSL/TLS certificate parameters, for example,
openssl.cnf, add the following information to the certificate’s Subject Alternative Name (SAN) field. For example:# ... [alt_names] <quay_registry_name>-quay-builder-<namespace>.<domain-name>:443 # ...For example:
# ... [alt_names] example-registry-quay-builder-quay-enterprise.apps.cluster-new.gcp.quaydev.org:443 # ...
Bare metal builds
Bare metal builds run Project Quay build workers on physical Red Hat Quay on OpenShift Container Platform or Kubernetes nodes so you can use existing hardware capacity for Dockerfile builds.
Use bare metal builds when you want builders on dedicated worker nodes and can accept container-level isolation rather than a full virtual machine per build. Plan for labeled worker nodes that can schedule build pods, a builder service account with the required permissions, and network access from builders to the build manager and to your Git sources.
Configure bare metal builder infrastructure when you are ready to implement builds; planning focuses on node capacity, isolation tradeoffs, and whether bare metal or virtual builders better match your security model.
Configuring bare metal builds for Red Hat Quay on OpenShift Container Platform
To configure bare metal builds for Red Hat Quay on OpenShift Container Platform with Project Quay, you can create a build project, configure service accounts, and update your configuration file.
|
Note
|
If you are using the Project Quay Operator on OpenShift Container Platform with a managed |
-
You have an OpenShift Container Platform cluster provisioned with the Project Quay Operator running.
-
You have set the
tlscomponent tounmanagedand uploaded custom SSL/TLS certificates to the Project Quay Operator. For more information, see SSL and TLS for Project Quay. -
You are logged into OpenShift Container Platform as a cluster administrator.
-
Enter the following command to create a project where Builds will be run, for example,
bare-metal-builder:$ oc new-project bare-metal-builder -
Create a new
ServiceAccountin thebare-metal-buildernamespace by entering the following command:$ oc create sa -n bare-metal-builder quay-builder -
Enter the following command to grant a user the
editrole within thebare-metal-buildernamespace:$ oc policy add-role-to-user -n bare-metal-builder edit system:serviceaccount:bare-metal-builder:quay-builder -
Enter the following command to retrieve a token associated with the
quay-builderservice account in thebare-metal-buildernamespace. This token is used to authenticate and interact with the OpenShift Container Platform cluster’s API server.-
If your OpenShift Container Platform cluster is version 4.11+, enter the following command:
oc create token quay-builder -n bare-metal-builder --duration 24h -
If your OpenShift Container Platform cluster is earlier than version 4.11, for example, version 4.10, enter the following command:
$ oc sa get-token -n bare-metal-builder quay-builder
-
-
Identify the URL for the OpenShift Container Platform cluster’s API server. This can be found in the OpenShift Container Platform web console.
-
Identify a worker node label to be used when scheduling build jobs. Because build pods must run on bare metal worker nodes, typically these are identified with specific labels.
Check with your cluster administrator to determine exactly which node label should be used.
-
Obtain the Kube API Server’s certificate authority (CA) to add to Project Quay’s extra certificates.
-
On OpenShift Container Platform versions 4.15+, enter the following commands to obtain the name of the secret containing the CA:
$ oc extract cm/kube-root-ca.crt -n openshift-apiserver$ mv ca.crt build_cluster.crt -
On OpenShift Container Platform versions earlier than 4.15, for example, 4.14, enter the following command:
$ oc get sa openshift-apiserver-sa --namespace=openshift-apiserver -o json | jq '.secrets[] | select(.name | contains("openshift-apiserver-sa-token"))'.name -
Obtain the
ca.crtkey value from the secret in the OpenShift Container Platform Web Console. The value begins with "-----BEGIN CERTIFICATE-----"`. -
Import the CA to Project Quay. Ensure that the name of this file matches the
K8S_API_TLS_CAfield used in Step 9.
-
-
Create the following
SecurityContextConstraintsresource for theServiceAccount:apiVersion: security.openshift.io/v1 kind: SecurityContextConstraints metadata: name: quay-builder priority: null readOnlyRootFilesystem: false requiredDropCapabilities: null runAsUser: type: RunAsAny seLinuxContext: type: RunAsAny seccompProfiles: - '*' supplementalGroups: type: RunAsAny volumes: - '*' allowHostDirVolumePlugin: true allowHostIPC: true allowHostNetwork: true allowHostPID: true allowHostPorts: true allowPrivilegeEscalation: true allowPrivilegedContainer: true allowedCapabilities: - '*' allowedUnsafeSysctls: - '*' defaultAddCapabilities: null fsGroup: type: RunAsAny --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: quay-builder-scc namespace: bare-metal-builder rules: - apiGroups: - security.openshift.io resourceNames: - quay-builder resources: - securitycontextconstraints verbs: - use --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: quay-builder-scc namespace: bare-metal-builder subjects: - kind: ServiceAccount name: quay-builder roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: quay-builder-scc -
Update the
config.yamlfile of your Red Hat Quay on OpenShift Container Platform deployment to include an appropriate bare metal builds configuration by using the OpenShift Container Platform web console.-
Click Operators → Installed Operators → Red Hat Quay → Quay Registry.
-
Click the name of your registry, for example, example-registry.
-
Under Config Bundle Secret, click the name of your configuration bundle, for example, extra-ca-certificate-config-bundle-secret.
-
Click Actions → Edit Secret.
-
Add the following information to your Project Quay
config.yamlfile, replacing each value with information that is relevant to your specific installation:FEATURE_USER_INITIALIZE: true BROWSER_API_CALLS_XHR_ONLY: false SUPER_USERS: - <superusername> FEATURE_USER_CREATION: false FEATURE_QUOTA_MANAGEMENT: true FEATURE_BUILD_SUPPORT: True BUILDMAN_HOSTNAME: ${BUILDMAN_HOSTNAME}:443 BUILD_MANAGER: - ephemeral - ALLOWED_WORKER_COUNT: 10 ORCHESTRATOR_PREFIX: buildman/production/ ORCHESTRATOR: REDIS_HOST: <sample_redis_hostname> REDIS_PASSWORD: "" REDIS_SSL: false REDIS_SKIP_KEYSPACE_EVENT_SETUP: false EXECUTORS: - EXECUTOR: kubernetes BUILDER_NAMESPACE: <sample_builder_namespace> K8S_API_SERVER: <sample_k8s_api_server> K8S_API_TLS_CA: <sample_crt_file> VOLUME_SIZE: 8G KUBERNETES_DISTRIBUTION: openshift CONTAINER_MEMORY_LIMITS: 1G CONTAINER_CPU_LIMITS: 300m CONTAINER_MEMORY_REQUEST: 1G CONTAINER_CPU_REQUEST: 300m NODE_SELECTOR_LABEL_KEY: beta.kubernetes.io/instance-type NODE_SELECTOR_LABEL_VALUE: n1-standard-4 CONTAINER_RUNTIME: podman SERVICE_ACCOUNT_NAME: <sample_service_account_name> SERVICE_ACCOUNT_TOKEN: <sample_account_token> QUAY_USERNAME: <quay_username> QUAY_PASSWORD: <quay_password> WORKER_IMAGE: <registry>/quay-quay-builder WORKER_TAG: <some_tag> BUILDER_VM_CONTAINER_IMAGE: registry.redhat.io/quay/quay-builder-qemu-rhcos-rhel8:v3.9.10-4 SETUP_TIME: 180 MINIMUM_RETRY_THRESHOLD: 0 SSH_AUTHORIZED_KEYS: - <ssh-rsa 12345 someuser@email.com> - <ssh-rsa 67890 someuser2@email.com> HTTP_PROXY: <http://10.0.0.1:80> HTTPS_PROXY: <http://10.0.0.1:80> NO_PROXY: <hostname.example.com>where:
BUILDMAN_HOSTNAME-
Specifies the hostname of the Project Quay registry. Obtain this by running the following command:
$ oc get route quayregistry-quay-builder -n ${QUAY_PROJECT} -o jsonpath='{.spec.host}'. BUILD_MANAGER.ORCHESTRATOR.REDIS_HOST-
Specifies the hostname for your Redis service.
BUILD_MANAGER.EXECUTORS.BUILDER_NAMESPACE-
Specifies the name of your bare metal builds namespace. This example used
bare-metal-builder. BUILD_MANAGER.EXECUTORS.K8S_API_SERVER-
Specifies the
K8S_API_SERVERis obtained by running$ oc cluster-info. BUILD_MANAGER.EXECUTORS.K8S_API_TLS_CA-
Specifies the name of your custom CA cert, for example,
K8S_API_TLS_CA: /conf/stack/extra_ca_certs/build-cluster.crt. BUILD_MANAGER.EXECUTORS.CONTAINER_MEMORY_LIMITS-
Specifies the memory limit for your container. Defaults to
5120Miif left unspecified. BUILD_MANAGER.EXECUTORS.CONTAINER_CPU_LIMITS-
Specifies the CPU limit for your container. Defaults to
1000mif left unspecified. BUILD_MANAGER.EXECUTORS.CONTAINER_MEMORY_REQUEST-
Specifies the memory request for your container. Defaults to
3968Miif left unspecified. BUILD_MANAGER.EXECUTORS.CONTAINER_CPU_REQUEST-
Specifies the CPU request for your container. Defaults to
500mif left unspecified. BUILD_MANAGER.EXECUTORS.SERVICE_ACCOUNT_TOKEN-
Specifies the token for your service account. Obtain this by running
$ oc create sa. BUILD_MANAGER.EXECUTORS.SSH_AUTHORIZED_KEYS-
Specifies the SSH authorized keys for your build environment. This key, or keys, should correspond to the private key that an admin or developer will use to SSH into the build worker for debugging purposes. This key can be obtained by establishing an SSH connection to the remote host using a specific SSH key and port. For example:
$ ssh -i /path/to/ssh/key/set/in/ssh_authorized_keys -p 9999 core@localhost.
-
-
Restart your Project Quay registry to enable the builds feature.
Builds limitations with managed routes
Project Quay builds have networking constraints when Red Hat Quay on OpenShift Container Platform uses managed routes. Plan for these constraints so build workers can reach the build manager.
OpenShift Container Platform routes typically serve traffic on a single port. Because builds use gRPC, the Operator creates a dedicated Route that directs that traffic to the build manager.
When you plan builds on OpenShift Container Platform, account for the following:
-
OpenShift ingress must support HTTP/2 for the gRPC protocol used by the build manager.
-
The build manager needs the build cluster CA certificate in the Project Quay configuration so workers can establish a secure connection.
-
Build jobs must resolve the build manager hostname. Custom subdomains require DNS that points to the OpenShift router.
See "Configuring Project Quay builds for managed routes" for implementation steps.
Configuring Project Quay builds for managed routes
To use Project Quay builds with managed routes and custom hostnames, you can configure DNS records and update your registry configuration. This enables gRPC communication between build executors and the build manager.
-
The Project Quay Operator is installed and a
QuayRegistryexists. -
Your
kubectlorocCLI tool is configured for the target cluster.
-
Enable HTTP/2 ingress on your OpenShift Container Platform cluster to support gRPC.
-
Retrieve the host address of the generated build-manager route:
$ kubectl get -n <namespace> route <quayregistry-name>-quay-builder -o jsonpath={.status.ingress[0].host} -
Create a CNAME record with your DNS provider that points your custom hostname (for example,
builder-registry.example.com) to the route host retrieved in the previous step. -
Update the
Secretreferenced byspec.configBundleSecretin yourQuayRegistryto include the build cluster CA certificate. The key must be namedextra_ca_cert_build_cluster.cert. -
Add the
BUILDMAN_HOSTNAMEfield to yourconfig.yamland include the port number:BUILDMAN_HOSTNAME: builder-registry.example.com:443 BUILD_MANAGER: - ephemeral ALLOWED_WORKER_COUNT: 1 ...
Virtual builds
Virtual builds in Project Quay run build workers in unprivileged containers on Red Hat Quay on OpenShift Container Platform. This approach provides process isolation without requiring a dedicated virtual machine for each build.
With virtual builds, the build manager creates a Kubernetes Job that starts a pod from the builder image. That image includes the builder binary and Podman. The pod runs unprivileged; the builder builds the image and reports status to the build manager.
Virtual builds limitations
The following limitations apply to virtual builds:
-
Running virtual builds in an unprivileged context might cause some Dockerfile commands that worked under a previous build strategy to fail. Changing build strategy can also affect build performance and reliability.
-
Running virtual builds directly in a container does not provide the same isolation as virtual machines. Changing the build environment might cause builds that previously succeeded to fail.
Configuring virtual builds for Red Hat Quay on OpenShift Container Platform
To configure virtual builds for Red Hat Quay on OpenShift Container Platform with Project Quay, you can create a build project, configure service accounts, and update your configuration file.
|
Note
|
|
-
You have an OpenShift Container Platform cluster provisioned with the Project Quay Operator running.
-
You have set the
tlscomponent tounmanagedand uploaded custom SSL/TLS certificates to the Project Quay Operator. For more information, see SSL and TLS for Project Quay. -
You have configured the OpenShift Container Platform TLS component for builds.
-
You are logged into OpenShift Container Platform as a cluster administrator.
-
Create a new project where your virtual builders will be run, for example,
virtual-builders, by running the following command:$ oc new-project virtual-builders -
Create a
ServiceAccountin the project that will be used to run builds by entering the following command:$ oc create sa -n virtual-builders quay-builderExample outputserviceaccount/quay-builder created -
Provide the created service account with editing permissions so that it can run a build:
$ oc adm policy -n virtual-builders add-role-to-user edit system:serviceaccount:virtual-builders:quay-builderExample outputclusterrole.rbac.authorization.k8s.io/edit added: "system:serviceaccount:virtual-builders:quay-builder" -
Grant the builder worker
anyuid sccpermissions by entering the following command. This requires cluster administrator privileges, which is required because builders must run as the Podman user for unprivileged or rootless builds to work.$ oc adm policy -n virtual-builders add-scc-to-user anyuid -z quay-builderExample outputclusterrole.rbac.authorization.k8s.io/system:openshift:scc:anyuid added: "quay-builder" -
Obtain the token for the builder service account by entering the following command:
$ oc create token quay-builder -n virtual-buildersNoteWhen the token expires you will need to request a new token. Optionally, you can also add a custom expiration. For example, specify
--duration 20160mto retain the token for two weeks.Example output<sample_account_token>... -
Determine the builder route by entering the following command:
$ oc get route -n quay-enterpriseExample outputNAME: example-registry-quay-builder HOST/PORT: example-registry-quay-builder-quay-enterprise.apps.stevsmit-cluster-new.gcp.quaydev.org PATH: SERVICES: example-registry-quay-app PORT: grpc TERMINATION: passthrough/Redirect WILDCARD: None -
Generate a self-signed SSL/TLS certificate with the
.crtextension by entering the following command:$ oc extract cm/kube-root-ca.crt -n openshift-apiserverExample outputca.crt -
Rename the
ca.crtfile tobuild-cluster.crtby entering the following command:$ mv ca.crt build-cluster.crt -
Update the
config.yamlfile of your Red Hat Quay on OpenShift Container Platform deployment to include an appropriate virtual builds configuration by using the OpenShift Container Platform web console.-
Click Operators → Installed Operators → Red Hat Quay → Quay Registry.
-
Click the name of your registry, for example, example-registry.
-
Under Config Bundle Secret, click the name of your configuration bundle, for example, extra-ca-certificate-config-bundle-secret.
-
Click Actions → Edit Secret.
-
Add an appropriate virtual builds configuration using the following as a reference:
FEATURE_USER_INITIALIZE: true BROWSER_API_CALLS_XHR_ONLY: false SUPER_USERS: - <superusername> FEATURE_USER_CREATION: false FEATURE_QUOTA_MANAGEMENT: true FEATURE_BUILD_SUPPORT: True BUILDMAN_HOSTNAME: <sample_build_route> BUILD_MANAGER: - ephemeral - ALLOWED_WORKER_COUNT: 1 ORCHESTRATOR_PREFIX: buildman/production/ JOB_REGISTRATION_TIMEOUT: 3600 ORCHESTRATOR: REDIS_HOST: <sample_redis_hostname> REDIS_PASSWORD: "" REDIS_SSL: false REDIS_SKIP_KEYSPACE_EVENT_SETUP: false EXECUTORS: - EXECUTOR: kubernetesPodman NAME: openshift BUILDER_NAMESPACE: <sample_builder_namespace> SETUP_TIME: 180 MINIMUM_RETRY_THRESHOLD: 0 BUILDER_CONTAINER_IMAGE: quay.io/projectquay/quay-builder:{producty} # Kubernetes resource options K8S_API_SERVER: <sample_k8s_api_server> K8S_API_TLS_CA: <sample_crt_file> VOLUME_SIZE: 8G KUBERNETES_DISTRIBUTION: openshift CONTAINER_MEMORY_LIMITS: 1G CONTAINER_CPU_LIMITS: 300m CONTAINER_MEMORY_REQUEST: 1G CONTAINER_CPU_REQUEST: 300m NODE_SELECTOR_LABEL_KEY: "" NODE_SELECTOR_LABEL_VALUE: "" SERVICE_ACCOUNT_NAME: <sample_service_account_name> SERVICE_ACCOUNT_TOKEN: <sample_account_token> HTTP_PROXY: <http://10.0.0.1:80> HTTPS_PROXY: <http://10.0.0.1:80> NO_PROXY: <hostname.example.com>where:
BUILDMAN_HOSTNAME-
Specifies that the build route is obtained by running
$ oc get route -nwith the namespace of your Red Hat Quay on OpenShift Container Platform deployment. A port must be provided at the end of the route, and it should use the following format:[quayregistry-cr-name]-quay-builder-[ocp-namespace].[ocp-domain-name]:443. BUILD_MANAGER.JOB_REGISTRATION_TIMEOUT-
Specifies that you might receive the following error when set too low:
failed to register job to build manager: rpc error: code = Unauthenticated desc = Invalid build token: Signature has expired. This parameter should be set to at least240. BUILD_MANAGER.ORCHESTRATOR.REDIS_HOST-
Specifies that you must update this field accordingly if your Redis host has a password or SSL/TLS certificates.
BUILD_MANAGER.EXECUTORS.BUILDER_NAMESPACE-
Specifies the name of your virtual builds namespace. This example used
virtual-builders. BUILD_MANAGER.EXECUTORS.K8S_API_SERVER-
Specifies the value obtained by running
$ oc cluster-info. BUILD_MANAGER.EXECUTORS.K8S_API_TLS_CA-
Specifies that you must manually create and add your custom CA cert, for example,
K8S_API_TLS_CA: /conf/stack/extra_ca_certs/build-cluster.crt. BUILD_MANAGER.EXECUTORS.CONTAINER_MEMORY_LIMITS-
Specifies the memory limit. Defaults to
5120Miif left unspecified. BUILD_MANAGER.EXECUTORS.CONTAINER_CPU_LIMITS-
Specifies that for virtual builds, you must ensure that there are enough resources in your cluster. Defaults to
1000mif left unspecified. BUILD_MANAGER.EXECUTORS.CONTAINER_MEMORY_REQUEST-
Specifies the memory request. Defaults to
3968Miif left unspecified. BUILD_MANAGER.EXECUTORS.CONTAINER_CPU_REQUEST-
Specifies the CPU request. Defaults to
500mif left unspecified. BUILD_MANAGER.EXECUTORS.SERVICE_ACCOUNT_TOKEN-
Specifies the token obtained when running
$ oc create sa.
Example virtual builds configurationFEATURE_USER_INITIALIZE: true BROWSER_API_CALLS_XHR_ONLY: false SUPER_USERS: - quayadmin FEATURE_USER_CREATION: false FEATURE_QUOTA_MANAGEMENT: true FEATURE_BUILD_SUPPORT: True BUILDMAN_HOSTNAME: example-registry-quay-builder-quay-enterprise.apps.docs.quayteam.org:443 BUILD_MANAGER: - ephemeral - ALLOWED_WORKER_COUNT: 1 ORCHESTRATOR_PREFIX: buildman/production/ JOB_REGISTRATION_TIMEOUT: 3600 ORCHESTRATOR: REDIS_HOST: example-registry-quay-redis REDIS_PASSWORD: "" REDIS_SSL: false REDIS_SKIP_KEYSPACE_EVENT_SETUP: false EXECUTORS: - EXECUTOR: kubernetesPodman NAME: openshift BUILDER_NAMESPACE: virtual-builders SETUP_TIME: 180 MINIMUM_RETRY_THRESHOLD: 0 BUILDER_CONTAINER_IMAGE: quay.io/projectquay/quay-builder:{producty} # Kubernetes resource options K8S_API_SERVER: api.docs.quayteam.org:6443 K8S_API_TLS_CA: /conf/stack/extra_ca_certs/build-cluster.crt VOLUME_SIZE: 8G KUBERNETES_DISTRIBUTION: openshift CONTAINER_MEMORY_LIMITS: 1G CONTAINER_CPU_LIMITS: 300m CONTAINER_MEMORY_REQUEST: 1G CONTAINER_CPU_REQUEST: 300m NODE_SELECTOR_LABEL_KEY: "" NODE_SELECTOR_LABEL_VALUE: "" SERVICE_ACCOUNT_NAME: quay-builder SERVICE_ACCOUNT_TOKEN: "<sample_account_token>" HTTP_PROXY: <http://10.0.0.1:80> HTTPS_PROXY: <http://10.0.0.1:80> NO_PROXY: <hostname.example.com> -
Click Save on the Edit Secret page.
-
-
Restart your Red Hat Quay on OpenShift Container Platform registry with the new configuration.
Modifying your AWS S3 storage bucket
To enable builds with AWS S3 storage in Project Quay, you can configure cross-origin resource sharing (CORS) settings in your S3 bucket. This allows build workers to access and store build artifacts in your S3 bucket.
-
Log in to your AWS console at s3.console.aws.com.
-
In the search bar, search for
S3and then click S3. -
Click the name of your bucket, for example,
myawsbucket. -
Click the Permissions tab.
-
Under Cross-origin resource sharing (CORS), include the following parameters:
[ { "AllowedHeaders": [ "Authorization" ], "AllowedMethods": [ "GET" ], "AllowedOrigins": [ "*" ], "ExposeHeaders": [], "MaxAgeSeconds": 3000 }, { "AllowedHeaders": [ "Content-Type", "x-amz-acl", "origin" ], "AllowedMethods": [ "PUT" ], "AllowedOrigins": [ "*" ], "ExposeHeaders": [], "MaxAgeSeconds": 3000 } ]
Modifying your Google Cloud Platform object bucket
To enable virtual builds with Google Cloud Platform storage in Project Quay, you can configure cross-origin resource sharing (CORS) settings in your GCP bucket. This allows build workers to upload Dockerfiles and access build artifacts.
|
Note
|
Currently, modifying your Google Cloud Platform object bucket is not supported on IBM Power and IBM Z. |
-
Use the following reference to create a JSON file for your specific CORS needs. For example:
$ cat gcp_cors.jsonExample output[ { "origin": ["*"], "method": ["GET"], "responseHeader": ["Authorization"], "maxAgeSeconds": 3600 }, { "origin": ["*"], "method": ["PUT"], "responseHeader": [ "Content-Type", "x-goog-acl", "origin"], "maxAgeSeconds": 3600 } ] -
Enter the following command to update your GCP storage bucket:
$ gcloud storage buckets update gs://<bucket_name> --cors-file=./gcp_cors.jsonExample outputUpdating Completed 1 -
You can display the updated CORS configuration of your GCP bucket by running the following command:
$ gcloud storage buckets describe gs://<bucket_name> --format="default(cors)"Example outputcors: - maxAgeSeconds: 3600 method: - GET origin: - '*' responseHeader: - Authorization - maxAgeSeconds: 3600 method: - PUT origin: - '*' responseHeader: - Content-Type - x-goog-acl - origin
Configure Clair database and custom Clair configuration
Configure unmanaged or Operator-managed Clair databases and supply custom clair-config.yaml settings for external databases, SSL/TLS certificates, and managed Clair deployments.
Unmanaged Clair configuration
Unmanaged Clair configuration lets you run a custom Clair deployment or use an external Clair database with the Project Quay Operator. You can use this option for geo-replicated environments or highly available databases outside your cluster.
Running a custom Clair configuration with an unmanaged Clair database
To use an external Clair database with the Project Quay Operator, you can set the clairpostgres component to unmanaged in your QuayRegistry custom resource.
|
Important
|
You must not use the same externally managed PostgreSQL database for both Project Quay and Clair deployments. Your PostgreSQL database must also not be shared with other workloads, as it might exhaust the natural connection limit on the PostgreSQL side when connection-intensive workloads, like Project Quay or Clair, contend for resources. Additionally, pgBouncer is not supported with Project Quay or Clair, so pgBouncer is not an option to resolve this issue. |
-
In the Quay Operator, set the
clairpostgrescomponent of theQuayRegistrycustom resource tomanaged: false:apiVersion: quay.redhat.com/v1 kind: QuayRegistry metadata: name: quay370 spec: configBundleSecret: config-bundle-secret components: - kind: objectstorage managed: false - kind: route managed: true - kind: tls managed: false - kind: clairpostgres managed: false
Configuring a custom Clair database with an unmanaged Clair database
To configure a custom Clair database with SSL/TLS certificates on Project Quay, you can create a Quay configuration bundle secret that includes the clair-config.yaml file.
|
Note
|
The following procedure configures Clair with SSL/TLS certificates. |
-
Create a Quay configuration bundle secret that includes the
clair-config.yamlby entering the following command:$ oc create secret generic --from-file config.yaml=./config.yaml --from-file extra_ca_cert_rds-ca-2019-root.pem=./rds-ca-2019-root.pem --from-file clair-config.yaml=./clair-config.yaml --from-file ssl.cert=./ssl.cert --from-file ssl.key=./ssl.key config-bundle-secretExample Clairconfig.yamlfileindexer: connstring: host=quay-server.example.com port=5432 dbname=quay user=quayrdsdb password=quayrdsdb sslrootcert=/run/certs/rds-ca-2019-root.pem sslmode=verify-ca layer_scan_concurrency: 6 migrations: true scanlock_retry: 11 log_level: debug matcher: connstring: host=quay-server.example.com port=5432 dbname=quay user=quayrdsdb password=quayrdsdb sslrootcert=/run/certs/rds-ca-2019-root.pem sslmode=verify-ca migrations: true metrics: name: prometheus notifier: connstring: host=quay-server.example.com port=5432 dbname=quay user=quayrdsdb password=quayrdsdb sslrootcert=/run/certs/rds-ca-2019-root.pem sslmode=verify-ca migrations: trueNote-
The database certificate is mounted under
/run/certs/rds-ca-2019-root.pemon the Clair application pod in theclair-config.yaml. It must be specified when configuring yourclair-config.yaml.
-
-
Add the
clair-config.yamlfile to your bundle secret, for example:apiVersion: v1 kind: Secret metadata: name: config-bundle-secret namespace: quay-enterprise data: config.yaml: <base64 encoded Quay config> clair-config.yaml: <base64 encoded Clair config> extra_ca_cert_<name>: <base64 encoded ca cert> ssl.crt: <base64 encoded SSL certificate> ssl.key: <base64 encoded SSL private key>NoteWhen updated, the provided
clair-config.yamlfile is mounted into the Clair pod. Any fields not provided are automatically populated with defaults using the Clair configuration module. -
You can check the status of your Clair pod by clicking the commit in the Build History page, or by running
oc get pods -n <namespace>. For example:$ oc get pods -n <namespace>Example outputNAME READY STATUS RESTARTS AGE f192fe4a-c802-4275-bcce-d2031e635126-9l2b5-25lg2 1/1 Running 0 7s
Running a custom Clair configuration with a managed Clair database
You can customize Clair settings while the Project Quay Operator manages the Clair database. Use this approach to disable updater resources or configure Clair for disconnected environments.
|
Note
|
|
Setting a Clair database to managed
To have the Project Quay Operator manage your Clair database, you can set the clairpostgres component to managed in your QuayRegistry custom resource.
-
In the Quay Operator, set the
clairpostgrescomponent of theQuayRegistrycustom resource tomanaged: true:apiVersion: quay.redhat.com/v1 kind: QuayRegistry metadata: name: quay370 spec: configBundleSecret: config-bundle-secret components: - kind: objectstorage managed: false - kind: route managed: true - kind: tls managed: false - kind: clairpostgres managed: true
Configuring a custom Clair database with a managed Clair configuration
To supply a custom clair-config.yaml while the Operator manages Clair on Project Quay, you can create a Quay configuration bundle secret that includes your Clair configuration file.
-
Create a Quay configuration bundle secret that includes the
clair-config.yamlby entering the following command:$ oc create secret generic --from-file config.yaml=./config.yaml --from-file extra_ca_cert_rds-ca-2019-root.pem=./rds-ca-2019-root.pem --from-file clair-config.yaml=./clair-config.yaml config-bundle-secretExample Clairconfig.yamlfileindexer: connstring: host=quay-server.example.com port=5432 dbname=quay user=quayrdsdb password=quayrdsdb sslmode=disable layer_scan_concurrency: 6 migrations: true scanlock_retry: 11 log_level: debug matcher: connstring: host=quay-server.example.com port=5432 dbname=quay user=quayrdsdb password=quayrdsdb sslmode=disable migrations: true metrics: name: prometheus notifier: connstring: host=quay-server.example.com port=5432 dbname=quay user=quayrdsdb password=quayrdsdb sslmode=disable migrations: trueNoteThe database certificate is mounted under
/run/certs/rds-ca-2019-root.pemon the Clair application pod in theclair-config.yaml. It must be specified when configuring yourclair-config.yaml. -
Add the
clair-config.yamlfile to your bundle secret, for example:apiVersion: v1 kind: Secret metadata: name: config-bundle-secret namespace: quay-enterprise data: config.yaml: <base64 encoded Quay config> clair-config.yaml: <base64 encoded Clair config>NoteWhen updated, the provided
clair-config.yamlfile is mounted into the Clair pod. Any fields not provided are automatically populated with defaults using the Clair configuration module. -
You can check the status of your Clair pod by clicking the commit in the Build History page, or by running
oc get pods -n <namespace>. For example:$ oc get pods -n <namespace>Example outputNAME READY STATUS RESTARTS AGE f192fe4a-c802-4275-bcce-d2031e635126-9l2b5-25lg2 1/1 Running 0 7s
Configure Clair updaters and disconnected scanning
Configure Clair updater sets and advanced per-updater settings, disable automatic updaters for disconnected deployments, and set up offline vulnerability data transfer and CPE mapping.
Clair updaters
Clair uses Go packages called updaters to fetch and parse vulnerability databases. You can control which databases Clair imports and how often vulnerability data is updated in Project Quay.
Updaters are usually paired with a matcher to interpret if, and how, any vulnerability is related to a package. Administrators might want to update the vulnerability database less frequently, or not import vulnerabilities from databases that they know are not used.
Selecting updater sets for full Red Hat Enterprise Linux (RHEL) coverage
You can select Clair updater sets to cover vulnerabilities in Red Hat Enterprise Linux (RHEL). Use the rhel, rhcc, clair.cvss, and osv updater sets for full coverage.
For full coverage of vulnerabilities in Red Hat Enterprise Linux (RHEL), you must use the following updater sets:
-
rhel. This updater ensures that you have the latest information on the vulnerabilities that affect RHEL. -
rhcc. This updater keeps track of vulnerabilities related to Red Hat’s container images. -
clair.cvss. This updater offers a comprehensive view of the severity and risk assessment of vulnerabilities by providing Common Vulnerabilities and Exposures (CVE) scores. -
osv. This updater focuses on tracking vulnerabilities in open-source software components. This updater is recommended due to how common the use of Java and Go are in RHEL products.
#...
updaters:
sets:
- rhel
- rhcc
- clair.cvss
- osv
#...
Advanced updater configuration
In some cases, users might want to configure updaters for specific behavior, for example, if you want to allowlist specific ecosystems for the Open Source Vulnerabilities (OSV) updaters.
Advanced updater configuration might be useful for proxy deployments or air-gapped deployments. Configuration for specific updaters in these scenarios can be passed by putting a key underneath the config environment variable of the updaters object. Users should examine their Clair logs to double-check names.
The following YAML snippets detail the various settings available to some Clair updaters.
|
Important
|
For most users, advanced updater configuration is unnecessary. |
Configuring the alpine updater
#...
updaters:
sets:
- alpine
config:
alpine:
url: https://secdb.alpinelinux.org/
#...
Configuring the debian updater
#...
updaters:
sets:
- debian
config:
debian:
mirror_url: https://deb.debian.org/
json_url: https://security-tracker.debian.org/tracker/data/json
#...
Configuring the clair.cvss updater
#...
updaters:
config:
clair.cvss:
url: https://nvd.nist.gov/feeds/json/cve/1.1/
#...
Configuring the oracle updater
#...
updaters:
sets:
- oracle
config:
oracle-2023-updater:
url:
- https://linux.oracle.com/security/oval/com.oracle.elsa-2023.xml.bz2
oracle-2022-updater:
url:
- https://linux.oracle.com/security/oval/com.oracle.elsa-2022.xml.bz2
#...
Configuring the photon updater
#...
updaters:
sets:
- photon
config:
photon:
url: https://packages.vmware.com/photon/photon_oval_definitions/
#...
Configuring the rhel updater
#...
updaters:
sets:
- rhel
config:
rhel:
url: https://access.redhat.com/security/data/oval/v2/PULP_MANIFEST
ignore_unpatched: true
#...
ignore_unpatched is a Boolean that indicates whether to include information about vulnerabilities that do not have corresponding patches or updates available.
Configuring the rhcc updater
#...
updaters:
sets:
- rhcc
config:
rhcc:
url: https://access.redhat.com/security/data/metrics/cvemap.xml
#...
Configuring the suse updater
#...
updaters:
sets:
- suse
config:
suse:
url: https://support.novell.com/security/oval/
#...
Configuring the ubuntu updater
#...
updaters:
config:
ubuntu:
url: https://api.launchpad.net/1.0/
name: ubuntu
force:
- name: focal
version: 20.04
#...
where:
updaters.config.ubuntu.force-
Specifies the inclusion of specific distribution and version details in the resulting UpdaterSet, regardless of their status in the API response. Useful when you want to ensure that particular distributions and versions are consistently included in your updater configuration.
updaters.config.ubuntu.force.name-
Specifies the distribution name that you want to force to be included in the UpdaterSet.
updaters.config.ubuntu.force.version-
Specifies the version of the distribution you want to force into the UpdaterSet.
Configuring the osv updater
#...
updaters:
sets:
- osv
config:
osv:
url: https://osv-vulnerabilities.storage.googleapis.com/
allowlist:
- npm
- pypi
#...
allowlist is the list of ecosystems to allow. When left unset, all ecosystems are allowed. Must be lowercase.
Disabling the Clair Updater component
You can disable the Clair updater component when you run Project Quay in a disconnected environment. Set matcher.disable_updaters to true in the Clair configuration.
In the following example, Clair updaters are disabled:
#...
matcher:
disable_updaters: true
#...
Configuring updaters
You can configure Clair updaters by using the updaters.sets key in clair-config.yaml. Use the following examples to select one or more updater sets for your Project Quay deployment.
|
Important
|
|
- Configuring Clair for multiple updaters
-
.Multiple specific updaters
#...
updaters:
sets:
- alpine
- aws
- osv
#...
- Configuring Clair for Alpine
-
.Alpine config.yaml example
#...
updaters:
sets:
- alpine
#...
- Configuring Clair for AWS
-
.AWS config.yaml example
#...
updaters:
sets:
- aws
#...
- Configuring Clair for Debian
-
.Debian config.yaml example
#...
updaters:
sets:
- debian
#...
- Configuring Clair for Clair CVSS
-
.Clair CVSS config.yaml example
#...
updaters:
sets:
- clair.cvss
#...
- Configuring Clair for Oracle
-
.Oracle config.yaml example
#...
updaters:
sets:
- oracle
#...
- Configuring Clair for Photon
-
.Photon config.yaml example
#...
updaters:
sets:
- photon
#...
- Configuring Clair for SUSE
-
.SUSE config.yaml example
#...
updaters:
sets:
- suse
#...
- Configuring Clair for Ubuntu
-
.Ubuntu config.yaml example
#...
updaters:
sets:
- ubuntu
#...
- Configuring Clair for OSV
-
.OSV config.yaml example
#...
updaters:
sets:
- osv
#...
Clair in disconnected environments
Clair supports disconnected Project Quay deployments that have no direct internet access. You can use the clairctl tool to transfer vulnerability database updates from an open host so Clair can scan images offline.
Clair uses a set of components called updaters to handle the fetching and parsing of data from various vulnerability databases. By default, updaters pull vulnerability data directly from the internet and work for immediate use.
|
Note
|
Currently, Clair enrichment data is CVSS data. Enrichment data is currently unsupported in disconnected environments. |
Setting up Clair in a disconnected OpenShift Container Platform cluster
To install the clairctl command-line utility on a disconnected OpenShift Container Platform cluster, you can copy the binary from a running Clair pod and make it executable.
-
Install the
clairctlprogram for a Clair deployment in an OpenShift Container Platform cluster by entering the following command:$ oc -n quay-enterprise exec example-registry-clair-app-64dd48f866-6ptgw -- cat /usr/bin/clairctl > clairctl -
Set the permissions of the
clairctlfile so that it can be executed and run by the user, for example:$ chmod u+x ./clairctl
Setting up a self-managed deployment of Clair for a disconnected OpenShift Container Platform cluster
To install the clairctl command-line utility for a self-managed Clair deployment on OpenShift Container Platform, you can copy the binary from a Clair container with Podman and make it executable.
-
Install the
clairctlprogram for a self-managed Clair deployment by using thepodman cpcommand, for example:$ sudo podman cp clairv4:/usr/bin/clairctl ./clairctl -
Set the permissions of the
clairctlfile so that it can be executed and run by the user, for example:$ chmod u+x ./clairctl
Common Product Enumeration mapping in Clair
Clair uses Common Product Enumeration (CPE) mapping files to map RPM packages to security data for Red Hat Enterprise Linux (RHEL) container images. Accurate vulnerability reports depend on these mapping files being available to the Clair scanner.
The scanner requires the CPE file to be present and accessible to process RPM packages properly. If these files are missing or inaccessible, RPM packages installed in the container image are skipped during the scanning process.
By default, the Clair indexer includes the repos2cpe and names2repos data files within the Clair container. This allows you to reference local paths such as /data/repository-to-cpe.json without additional external configuration.
|
Important
|
While Red Hat Product Security updates CPE files regularly, the versions bundled within the Clair container are only updated during Project Quay releases. This can lead to temporary discrepancies between the latest security data and the versions bundled with your current installation. |
CPE mapping configuration reference
Common Product Enumeration (CPE) mapping configuration defines the fields and file paths used by Clair to associate packages with standardized product identifiers.
| CPE Type | Link to JSON mapping file |
|---|---|
|
|
|
indexer:
scanner:
repo:
rhel-repository-scanner:
repo2cpe_mapping_file: /data/repository-to-cpe.json
package:
rhel_containerscanner:
name2repos_mapping_file: /data/container-name-repos-map.json
where:
repo2cpe_mapping_file-
Specifies the path to the JSON file mapping Red Hat repositories to CPEs.
name2repos_mapping_file-
Specifies the path to the JSON file mapping container names to repositories.