Terraform organization — Part I : What if you split your components ?

Hey everyone, I hope you are doing well.

Today’s post will be about the organization of Terraform Repository.

Recently, I had a huge number of resources to deploy using Terraform, we did it in a monothic way. In another words we had something like :

As a result of this, we got a huge state file. It may be ok if we had a small project without a lot of components. But since we got an important infrastructure, believe me, we had some problems.

• A huge state file where the refresh may take an eternity, imagine if your connection fails, you will restart the refresh !
• Many API calls to the provider, which may cause some timeout errors and you may reach the provider’s quota;
• A wrong operation (Yes I am talking about terraform destroy) will destroy the whole Infrastructure ! You may find yourself with some components (so some resources) that you don’t have to manage, and you can break these components which are not yours.

What if you split your Terraform by components (knowing that you’ve already splitted by environment) ? What does it mean ? Just look at the schema below :

Do you know microservices ? Well it’s the same thing but using Terraform. The idea is, instead of having a huge state file, we will split it into small state-files so small components that will give us more advantage on our infrastructure .

Using the splitted way, you will get benefit of a lot of things, if you want to update the network component by example, you don’t have to check the whole infrastructure just for this update. The split will give each team its component to manage instead of having a heavy huge state.

I know that using this way, it is a little be harder to manage the modules and the state-files but at least you will win more safety and more improvements in your operations.

In the next post of this serie, we will talk about splitting the infrastructure by environment and components. The last one will be about managing the state files and a demo that recaps the serie.

Stay tunned,