TAP 1.6 – Crossplane Updates

Crossplane has been updated to version 1.12.1 in TAP 1.6 and this bring along some really amazing features! Beyond the bump of crossplane which we will discuss in length bellow, a few more fixes and additions were made to the TAP packaging of Crossplane to improve the UX. These updates include the support for installing providers in environments with custom CA certificates, ability to configure if to orphan or delete all crossplane resources and XRDs when deleting the package installation, support for working with an externally installed crossplane implementation via Helm or other means, and finally improved package configuration to make the package installation wait for the Crossplane providers to be ready and healthy before completing. With that all behind us, lets get the real exiciting features that have been unlocked, with the update to Crossplane 1.12. While the updates in Crossplane 1.12 are huge, I want to focus here on the 2 key features which can greatly improve the integration within TAP! Observe Only Resources While I am a huge fan of Crossplane, in the world of IaC, Terraform is probably the most common tool used. When evaluating Crossplane and Terraform, each has its pros and cons, but one of the things we saw as a huge challenge for crossplane was the fact that they did not have a mechanism similar to a data source in terraform. Crossplane only knew about objects it managed. this made integrations in public clouds in particular a very difficult task. If i want to create an RDS instance in an existing VPC, i want to be able to pull out the VPC details that i need from the cloud, and pull out the needed subnet details as well, and then use them in my resource i want to create. Previous to Crossplane 1.12, you needed to pass in any values to your resources manually, and their was no dynamic lookup mechanism available. This has now changed and we now have a great new feature called Observe Only Resources (OOR)! With OOR, Crossplane is able to observe and expose the full live state of an external resource, without performing any write or destructive operations. This can be read about in depth, including the design decisions and more in the following link. This opens up amazing capabilities and allows for more straight forward and production ready compositions to be made for real world scenarios in the TAP world for backing services in the different cloud providers! I am already working on some interesting use cases, and plan to share some of my new examples in the near future on github. The next major change in Crossplane 1.12 that can strongly benefit TAP is the introduction of provider families. Provider Families Crossplane is amazing, and offers great valkue, especially when integrating with public cloud services, however till now their were serious issues in terms of performance due to the large number of CRDs installed by Crossplane providers. When you for example installed the AWS provider in TAP 1.5, that would install 850+ CRDs to your cluster, which in the best of cases slowed down your cluster, and in some cases could cause your API server to crash if it was not sized correctly for such load! The Crossplane team understood this issue was serious, and began working on this from multiple directions. The first and best approach taken, was to go to the upstream kubernetes community and try to work on better scalability of the parts of the API server in charge of managing CRDs. While this work is progressing, and improvements are being made, the process is slow, and will take a long time to role out to all environments. The next approach is what we have here now in Crossplane 1.12, which is the idea of Provider Families. With provider families the idea is to break up the old monolithic providers into smaller, service based providers, and then depending on which resources you need to manage, you only install the providers that you need. The AWS provider for example has been broken into 155 different providers. Lets take the example of the Official AWS Provider from upbound which previously installed 903 CRDs into your cluster!!! If lets say we need to manage RDS instances, VPCs, and IAM roles, we would now need the following providers: provider-aws-iam – 22 CRDs provider-aws-vpc – 1 CRD provider-aws-rds – 21 CRDs This means that for this type of environment you would go from 903 down to 44 CRDs!!! This is a huge improvement, and it enables us to truly build our solutions as we need them, without putting unneeded stress on our clusters. Summary The new version of Crossplane, truly unlocks a huge set of use cases for advanced service bindings, allowing for maximum control, with maximum DevEx, and maximum performance all at the same time!