Hi all! Loving the discussions here 🙂 I have a question myself: How to deal with internal Terraform modules versioning in a monorepo? I love that you can use tags on modules in Terraform using git, so you can use semver to allow modules to be updated in a nice way. However, we've got to a point where we moved from one repo per module to a single mono repo with all the modules, so that maintenance is easier (and we don't have a bunch of repos with small Terraform code). The problem with this is that now you cannot really use tags with semver to track the modules, as they have different lifecycles. How do you guys deal with this? Duplicate code and create a folder per "active version" of the module? Some variation of trunk-based development? Just give up on monorepo and use a repo per module? Thanks in advance 😄

Isn’t one of the goals of a monorepo to ensure that all packages/modules gets the same dependency version? so you won’t have divergent packages?

To be honest I'm not too versed on monorepos. Can you explain this concept better @Khaled Ezzughayyar? For example, to me it makes sense that we might want to choose to make a breaking change in a module that is used in 3 places, refactor it in 2 and leave 1 using an older version. Would monorepo "philosophy" be against this?

Refer to the githash version of the module (so even though its within the same repo, its treated as an external module and therefore is cloned at a specific hash vs using that which is on trunk) Once all dependancies have updated to use the version on trunk, the git aspect would fall away

@Thameez but in this case I couldn't use semver, right? For example, I like the possibility of using

ref=v2.0

and if we update from

2.0.1

to

2.0.2

the update is propagated to those who use

ref=v2.0

instead of

ref=v2.0.1

. Same with using major as a ref, etc...

Hot Take: The best dependency management tool (for any language) is

git

In a monorepo: • How do I get the latest version of everything?:

git pull

• How do I make sure that my feature branch is up to date with all my (internal) dependencies?:

git merge origin/main

• How do I browse all the code that is in production?:

git checkout main

• What changed in this commit/feature branch/etc?:

git diff

Since you have all of this in one repo, any time you make a change you can to fix all the callers, and that change is atomic (for everyone else) because it can all be in one commit (or one merge). Having good CI also helps to find any callers that you forgot about or didn't even know about. I would highly recommend against any "fancy"/"clever" versioning, just use

git

. 😄

I wouldn’t say I’m the best to talk about the “monorepo” philosophy. But one of the main drivers for me to choose monorepo over multi-repo, is I don’t need to handle each dependency’s version alone in different changes. Which makes my changes as @Nicholas Capo said: “atomic”. So I would yes. the “philosophy” of monorepo is against this

Thanks for the input @Khaled Ezzughayyar and @Nicholas Capo. One thing that I'm having trouble is to fully grasp how this would work within a complex environment. Every change to my module would also mean a change to all other codes that refer to it? What if I don't want that? Going to more practical terms, should I use a "hash" to "lock" the version I use? Doesn't this require me to remember to update these everytime? Or I shouldn't lock the versions at all and just use the latest version of the module? If so, that seem a bit unpractical to me, as sometimes I can make breaking changes that I don't really want to deal with right now. Thanks 😄

Every change to my module would also mean a change to all other codes that refer to it? What if I don't want that?

If necessary (specifically a breaking change) then yes. But deferring those updates until later (never?) makes them even harder to do later. If every change to your module is immediately used by all the plans, then finding/reporting bugs becomes easier, and there are never any old versions to maintain, only

main

.

Doesn't this require me to remember to update these everytime?

If you use

git

as the dependency manager then locking the versions will make things extra hard for exactly the reason your mention.

I can make breaking changes that I don't really want to deal with right now

I would avoid adding the tech debt 😉 Whats the point of making a change if no one will use it? Does software without a user really exist? Can you run

terraform plan

in CI (or locally)? That will give you confidence in your change being correct (for all users).

If I understand correctly, you need a way to be able to refer to specific versions of the individual modules? One way to achieve this is to have some process (probably a Github Action) does the following on a push to the main branch: • identifies exactly which module has changed (from the file path) • creates a tag with the module name in it, and the new version: e.g.

my-module-1.0.1

. • you can then refer to that tag specifically when consuming the module. There is a downside to this method though: every

terraform init

will pull the entire Git repo and store it in a local

.terraform

directory. Although I think that may be unavoidable if you are using a single repo, either way. The 'proper' way to do this, if you want to avoid CD-ing the code on every commit, is to use a Terraform registry to register the modules.

Are you using a module registry? If you intend to then I would not waste effort on making a monorepo work

but in this case I couldn't use semver, right?

Not if your commits have a semver tag attached, The response by Louis above encapsulates it perfectly (including the downside)

What scale are you working with that mono repo is beneficial? This must only be for small team?

Thanks for the input all! We are currently a small Platform squad (3 people). We don't use trunk-based development for now (but intend to move to in the near future) or a module registry. @Nicholas Capo I very much like the idea of using the same version across all modules and know its benefits, but sometimes it is unfeasible for us 🙃 E.g. a module refactoring makes it so that existing resources are recreated, or a very long process of state migration. How would you deal with, for example, a refactoring that works on 99% of the code, but on one specific env it breaks naming convention and that leads to resources being recreated? Adapting the new code for this specific case feel detrimental to a refactoring, but also leaving outdated legacy code feels a bad idea haha @Louis McCormack I like your idea! Feels a bit complex but understandable haha Given you guys suggestion it seems like using a module registry is a possible solution to no worry about this? I saw that Hashicorp provides a "free" (is it truly free?) one. How usable is it? Or should we go for a self-hosted one? I looked at the available ones and the "biggest" has around 200 stars on GitHub, which makes me a bit anxious of using it. We use Azure DevOps for Git and artifacts registry. Looking at their documentation it seems like they don't provide a module registry as GitHub or GitLab does

How would you deal with, for example, a refactoring that works on 99% of the code, but on one specific env it breaks naming convention and that leads to resources being recreated?

Yeah, I admit, that's not an easy problem to solve 😕