Skip to content

kubernetes driver // allow custom annotations and labels - #1938

Merged
crazy-max merged 1 commit into
docker:masterfrom
jonapich:feature/custom-annotations
Jul 19, 2023
Merged

crazy-max merged 1 commit into
docker:masterfrom
jonapich:feature/custom-annotations

Conversation

@jonapich

@jonapich jonapich commented Jul 12, 2023 •

Copy link
Copy Markdown
Contributor

This implements custom annotations and labels to be included in kubernetes manifests. Currently, the user has to patch them in a subsequent call.

Motivation: We provide a tool for developers to create/remove buildx builders on a shared k8s cluster. To prevent wasting resources, the buildx deployments are patched with a label (for filtering) and an annotation (an expire-after timestamp). A cron janitor eventually finds and removes them. The user may also refresh the timestamp in order to retain their cache for longer if they desire.

Issue: Because creating the builder and patching the annotations requires two calls, I need to handle the cases when the create call fails. This isn't always possible in an exception handler, for instance when this happens on a CI server that gets killed abruptly. This forces the janitor to include additional heuristics to find and remove the leaked deployments. It would be simpler if these annotations would be included from the start.

@jonapich
jonapich marked this pull request as ready for review July 13, 2023 11:34
Comment thread docs/reference/buildx_create.md Resolved
Comment thread docs/reference/buildx_create.md Resolved
Comment thread driver/kubernetes/factory_test.go Resolved
Comment thread docs/reference/buildx_create.md Outdated
Passes additional driver-specific options.

Note: When using quoted values for example for the `nodeselector` or
Note: When using quoted values for example for the `nodeselector`, `annotations` or

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Note: When using quoted values for example for the `nodeselector`, `annotations` or
Note: When using quoted values for example for the `nodeselector`, `annotations`, `labels` or

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was expecting this comment :)

Should I remove for example from the sentence, since it's supposed to be all-inclusive?

Suggested change
Note: When using quoted values for example for the `nodeselector`, `annotations` or
Note: When using quoted values for the `nodeselector`, `annotations`, `labels` or

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(note: I applied my suggestion)

Comment thread driver/kubernetes/factory.go Resolved
Comment thread driver/kubernetes/factory_test.go Outdated
Comment on lines +218 to +220

_, _, _, err := f.processDriverOpts(cfg.Name, "test", cfg)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: remove extra lines

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I applied this comment on the other identical surrounding tests as well. I also noticed an extra ns := "test" at the beginning that was not necessary since it was rewritten in the next call.

@jonapich
jonapich requested a review from crazy-max July 19, 2023 11:43
Co-authored-by: Akihiro Suda <suda.kyoto@gmail.com>

Signed-off-by: Jonathan Piché <jpiche@coveo.com>

@crazy-max crazy-max left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants