Conductor is a tool that allows you to gain the advantages of separate components without the downsides of having multiple repositories. Since the long needed arrival of Composer in the PHP world, having one repository per package has been propelled into mainstream and often used without any questioning.
For product development, in which different applications depend on internal packages - which is our case at MyBuilder.com - having a repository for each package caused a coordination overhead that slowed down development. It just didn’t work for us.
Imagine you have three applications (admin, API and frontend) that depend on the same internal package and this package keeps changing. And every time the package changes you also have to update those three applications.
Because of that we had to switch back to a monolith repository and developed a tool to fix the problem of working
composer.json files in it.
We do not advocate against any approach and in fact we think both should be used. But if you are entangled in the above situation this post is perfect for you! :)
Let’s explore the problem more deepily, so we can better understand it:
Multiple repository approach
This approach is good for open-source projects, specially when contribution and versioning is important.
Check out this post from Matthew Weier O’Phinney, it’s excellent in explaining it.
Single repository approach
In a product development perspective, sharing code and having the latest version of your packages is mandatory as they contain business rules that change fast. In a single repository you also can unify tests and continuous integration (CI) in one tool and create single pull requests that affect more than one application when change occurs.
Not long ago in a distant galaxy
Now that we explained the problem really well (hopefully), let’s talk a little bit about how we solved this problem.
Composer doesn’t support multiple
composer.json files in a single repository and we needed it to maintain
isolation (that’s why we use packages daaaahhh). We want everything in one place but still isolated! Interesting problem!!
We also wanted to make single pull requests when packages changed, specially if the changes impacted different applications. We wanted everything documented in one place.
And also we wanted unified tooling for tests and CI.
Oh gosh, and for us only the latest version of our packages matter, because our business rules change frequently.
With those problems in mind we decided to created the Conductor .
What is Conductor
Conductor is a console application, which can be seen as a tool to help Composer manage internal packages.
When is it needed?
Conductor is needed when you have a single repository but wish to depend on multiple internal packages each with their own
How conductor solves the problem
We realised that we could use Composer’s ‘artifact’ repository type as our internal packages source, but we cheated
a bit as instead of zipping the package’s source code into the
artifact folder we just put instructions for Conductor and Composer in there.
Another problem that came across was that Composer generates absolute paths in the lock file even when we have given relative path to the artifact directory.
We run our applications in different environments (i.e developer machines, dev server, production server),
so changing the
composer.lock absolute paths in the artifact files to relative paths was necessary.
Another thing was that it didn’t feel right having multiple copies of the packages in the same repository so to fix it we symlink the packages in the
vendor folder to their source folder, that’s another important Conductor feature.
And now let’s see in step-by-step fashion how Conductor works:
- It uses Composer artifact repository type as an internal packages source.
composer updateis run only info about all the dependent packages are zipped and archived in the
- When running
composer install, composer find the package’s
composer.jsonfile in zipped package file the
artifact/folder so it can resolve the dependencies.
- After running
composer updateConductor then re-writes the
composer.lockfile to use relative paths to the
- Before Composer dumps the autoloader, Conductor symlinks the packages in the
As you can see in the above image, if we didn’t have Conductor we would need to update five repositories if we ever needed to change Package2.
The benefits of using Conductor
Now with Conductor everything is fantastic! You get all the benefits for having everything in a single repository but not losing any for having isolated packages. Also we can now do single pull requests and use uniform tooling for tests and CI, what a dream!!