Skip to main content

idpBuilder on Local Machine

About

WORK IN PROGRESS: This tool is in a pre-release stage and is under active development.

Spin up a complete internal developer platform using industry standard technologies like Kubernetes, Argo, and backstage with only Docker required as a dependency.

This can be useful in several ways:

  • Create a single binary which can demonstrate an IDP reference implementation.
  • Use within CI to perform integration testing.
  • Use as a local development environment for IDP engineers.

Prerequisites

A container engine is needed locally, such as:

NameSupportedRemark
Docker desktopYes
Podman desktopNoidpbuilder can create a cluster using podman rootful
FinchNo

Note: Set the DOCKER_HOST env var property using podman to let idpbuilder to talk with the engine (e.g export DOCKER_HOST="unix:///var/run/docker.sock")

Quick Start

You can execute the following bash script to get started with a running version of the idpBuilder (inspect the script first if you have concerns):

danger
curl -fsSL https://raw.githubusercontent.com/cnoe-io/idpbuilder/main/hack/install.sh | bash

verify a successful installation by running the following command and inspecting the output for the right version:

idpbuilder version

Alternatively, you can run the following commands for a manual installation:

version=$(curl -Ls -o /dev/null -w %{url_effective} https://github.com/cnoe-io/idpbuilder/releases/latest)
version=${version##*/}
curl -L -o ./idpbuilder.tar.gz "https://github.com/cnoe-io/idpbuilder/releases/download/${version}/idpbuilder-$(uname | awk '{print tolower($0)}')-$(uname -m | sed 's/x86_64/amd64/').tar.gz"
tar xzf idpbuilder.tar.gz

./idpbuilder version
# example output
# idpbuilder 0.4.1 go1.21.5 linux/amd64

Or, you can download the latest binary from the release page.

tip

If you are interested in running idpbuilder in Codespaces through your browser, check out the Codespaces section.

Using the idpbuilder

Basic usage

The most basic command which creates a Kubernetes Cluster (Kind cluster) with the core packages installed.

./idpbuilder create
What are the core packages?
  • ArgoCD is the GitOps solution to deploy manifests to Kubernetes clusters. In this project, a package is an ArgoCD application.

  • Gitea server is the in-cluster Git server that ArgoCD can be configured to sync resources from. You can sync from local file systems to this.

  • Ingress-nginx is used as a method to access in-cluster resources such as ArgoCD UI and Gitea UI.

    Core package versions

    NameVersion
    Argo CDv2.10.7
    Giteav9.5.1
    Nginxv1.8.1

    The default manifests for the core packages are available here. See the contribution doc for more information on how core packages are installed and configured.

Once idpbuilder finishes provisioning cluster and packages, you can access GUIs by going to the following addresses in your browser.

You can obtain credentials for them by running the following command:

./idpbuilder get secrets
The "get secrets" command

The get secrets command retrieves the following:

  • ArgoCD initial admin password.

  • Gitea admin user credentials.

  • Any secrets labeled with cnoe.io/cli-secret=true.

    You can think of the command as executing the following kubectl commands:

    kubectl -n argocd get secret argocd-initial-admin-secret
    kubectl get secrets -n gitea gitea-admin-secret
    kubectl get secrets -A -l cnoe.io/cli-secret=true

    In addition, secrets labeled with cnoe.io/package-name can be specified with the -p flag. For example, for Gitea:

    ./idpbuilder get secrets -p gitea

Example commands

For more advanced use cases, check out the Stacks Repository.

You can specify the kubernetes version by using the --kube-version flag. Supported versions are available here.

./idpbuilder create --kube-version v1.27.3

If you want to specify your own kind configuration file, use the --kind-config flag.

./idpbuilder create --build-name local --kind-config ./my-kind.yaml`

If you want to specify ArgoCD configmap.

./idpbuilder create --package-custom-file=argocd:pkg/k8s/test-resources/input/argocd-cm.yaml

Run the following commands for available flags and subcommands:

./idpbuilder --help
./idpbuilder create --help

Custom Packages

Idpbuilder supports specifying custom packages using the flag --package-dir flag. This flag expects a directory (local or remote) containing ArgoCD application files and / or ArgoCD application set files. In case of a remote directory, it must be a directory in a git repository, and the URL format must be a kustomize remote URL format.

Examples of using custom packages are available in the stacks repository. Let's take a look at this example. This defines two custom package directories to deploy to the cluster.

To deploy these packages, run the following command:

./idpbuilder create \
--package-dir https://github.com/cnoe-io/stacks//basic/package1 \
--package-dir https://github.com/cnoe-io/stacks//basic/package2

Alternatively, you can use the local directory format.

# clone the stacks repository
git clone https://github.com/cnoe-io/stacks.git
cd stacks
# run idpbuilder against the local directory
./idpbuilder create \
--package-dir examples/basic/package1\
--package-dir examples/basic/package2

Running this command should create three additional ArgoCD applications in your cluster.

$ kubectl get Applications -n argocd  -l example=basic
NAME SYNC STATUS HEALTH STATUS
guestbook Synced Healthy
guestbook2 Synced Healthy
my-app Synced Healthy

Let's break this down. The first package directory defines an application. This corresponds to the my-app application above. In this application, we want to deploy manifests from local machine in GitOps way.

The directory contains an ArgoCD application file. This is a normal ArgoCD application file except for one field.

apiVersion: argoproj.io/v1alpha1
kind: Application
spec:
source:
repoURL: cnoe://manifests

The cnoe:// prefix in the repoURL field indicates that we want to sync from a local directory. Values after cnoe:// is treated as a relative path from this file. In this example, we are instructing idpbuilder to make ArgoCD sync from files in the manifests directory.

As a result the following actions were taken by idpbuilder:

  1. Create a Gitea repository.
  2. Fill the repository with contents from the manifests directory.
  3. Update the Application spec to use the newly created repository.

You can verify this by going to this address in your browser: https://gitea.cnoe.localtest.me:8443/giteaAdmin/idpbuilder-localdev-my-app-manifests

img.png

This is the repository that corresponds to the manifests folder. It contains a file called alpine.yaml, synced from the manifests directory above.

You can also view the updated Application spec by going to this address: https://argocd.cnoe.localtest.me:8443/applications/argocd/my-app

myapp

The second package directory defines two normal ArgoCD applications referencing a remote repository. They are applied as-is.

Contributing

If you'd like to contribute to the project or know the architecture and internals of this project, check out the contribution doc.

Running in Codespaces

Create a Codespaces instance. img 2. Wait for it to be ready. It may take several minutes. 3. Get the latest release of idpbuilder:

 version=$(curl -Ls -o /dev/null -w %{url_effective} https://github.com/cnoe-io/idpbuilder/releases/latest)
version=${version##*/}
curl -L -o ./idpbuilder.tar.gz "https://github.com/cnoe-io/idpbuilder/releases/download/${version}/idpbuilder-$(uname | awk '{print tolower($0)}')-$(uname -m | sed 's/x86_64/amd64/').tar.gz"
tar xzf idpbuilder.tar.gz
  1. Run idpbuilder:
     idpbuilder create --protocol http  \
    --host ${CODESPACE_NAME}-8080.${GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN} \
    --port 8080 --use-path-routing
  2. Because Codespaces gives a single externally routable host name for an instance, idpbuilder must deploy with path based routing. This means ArgoCD and Gitea UIs are given with the following commands.
    • ArgoCD: echo https://${CODESPACE_NAME}-8080.${GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN}/argocd
    • Gitea: echo https://${CODESPACE_NAME}-8080.${GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN}/gitea
  3. Note that not all examples work with path based routing.

Extending the IDP builder

We are actively working to include more patterns and examples of extending idpbuilder to get started easily.