In June 2021, we introduced Collectives™ on Stack Overflow. Collectives are a set of spaces on Stack Overflow where content related to certain technical languages, products, practices, or services are grouped together. It’s a place for users who regularly interact with these technologies to collaborate and for the organizations who help build or maintain this technology to share their expertise. When we first launched Collectives, providers such as Google Cloud, Twilio, and AWS (see the full list here) identified subject matter experts from their teams to contribute knowledge, validate great answers provided by the community, and recognize users for their contributions. They continue to do this today.
In February of this year, we launched our first topic-focused collectives, expanding the product to include areas of practice. In addition to collectives focusing on a technology managed by a specific organization, these collectives are organized around a topic like CI/CD or a language like R. Earlier this month, we released Discussions within several collectives, a new feature that lets users have open-ended discussions, the kind of subjective conversation that isn’t right for traditional Stack Overflow Q&A, but that can still spread knowledge and provide value to our community.
Today, we wanted to showcase how this new feature is adding value to these communities of practice by sharing a recent Discussion post from the CI/CD Collective. If you find the conversation below interesting, you can join to add your voice to the discussion.
—
User GuiFalourd kicked things off with this Discussion post.
I've recently read a great article—Rethinking infrastructure as code from scratch—from Nathan Peck (Senior Developer Advocate at AWS).
There, he questioned the infrastructure complexity and the current state of infrastructure as code.
We’ll probably see a lot of innovation in the infrastructure as code space in the near future. What do you expect to see in the future to reduce infrastructure as code complexity?
Why? (Quoting Nathan below)
Cloud will not get simpler: It’s easy to start out with a simple service that has only a few features that can be configured in a few opinionated ways. But this type of service will never reach the same level of mass adoption as a lower level building block that is more configurable.
Infrastructure as code will not get simpler: infrastructure as code cannot get significantly simpler in its current form, because the underlying cloud is complex and only growing more complex.
I personally liked the idea of Nathan in his article, to rethink infrastructure as code declarations. I wonder if this is the best path to follow or if we could imagine other alternatives, as well as how it could influence CI/CD configurations in the future.
Users offered up a number of great responses, including this one from deric4:
I'm admittedly just getting up to date with the current status of IaC after playing Ron Swanson for the last couple years after resigning from my devops/platform engineer role so this may be a bad take :D .
"We'll probably see a lot of innovation in the infrastructure as code space in the near future. "
I definitely think we'll see marked improvements over what's available now, but I think "innovation" might not be the most apt descriptor.
There's a non trivial amount of risk for individuals / companies to invest in building products in the Infrastructure as Code space when the APIs they inherently depend on are controlled by multiple vendors who have shown that they will "compete" in the same space (i.e. terraform , pulumi, serverless framework vs cloudformation, aws-sam-cli, cdk, arm/bicep).
From the article: “Unfortunately existing infrastructure as code techniques are struggling to keep up with the challenge of defining these modern cloud resources because the underlying infrastructure as code languages are still using an approach similar to 30 year old HTML."
I agree that IaC techniques are struggling to keep up, but I don't think it has anything to do with the IaC languages/DSLs themselves. It's the limited ability to actually observe and manipulate the state of the infrastructure, which is at the mercy of the black box on the other side of the limited/opaque API wall see aws-sdk github issues . At least for AWS, no matter how good the IaC language/DSL is, its still going to be restricted by the capabilities of the the public APIs or the ones internal to CloudFormation.
Until you can query your cloud API to dump every resource that's been created and its state in your AWS account across all regions with out making hundreds/thousands of api calls, the only improvements we'll likely see are GUIs representing connections between resources and better DX for green field IaC. Day 2 operations and IaC changes to long living deployments will still be the pain points, much as it seems like it is today.
There are a number of other viewpoints within the Discussion, not all of which agree with one another — but that’s the point of the new feature: to encourage dialogue, debate, and learning through a great exchange of ideas.
Discussions are available in the collectives below. Be sure to check them out if you are interested in these topics and feel free to leave your thoughts about the potential for this format as a comment on this blog.